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DEFENSE  ACQUISITIONS 

Production  and  Fielding  of  Missile  Defense 
Components  Continue  with  Less  Testing  and 
Validation  Than  Planned 


Why  GAO  Did  This  Study 


What  GAO  Found 


The  Missile  Defense  Agency  (MDA) 
has  spent  about  $56  billion  and  will 
spend  about  $50  billion  more 
through  2013  to  develop  a  Ballistic 
Missile  Defense  System  (BMDS). 
GAO  was  directed  to  assess  the 
annual  progress  MDA  made  in 
developing  the  BMDS  as  well  as 
improvements  in  accountability 
and  transparency  in  agency 
operations,  management  processes, 
and  the  new  block  strategy.  To 
accomplish  this,  GAO  reviewed 
contractor  cost,  schedule,  and 
performance;  tests  completed;  and 
the  assets  fielded  during  2008.  GAO 
also  reviewed  pertinent  sections  of 
the  U.S.  Code,  acquisition  policy, 
and  the  activities  of  the  new  Missile 
Defense  Executive  Board  (MDEB). 
An  appendix  on  the  effect  the 
cancellation  of  a  Ground-based 
Midcourse  Defense  flight  test 
(FTG-04)  had  on  BMDS 
development  is  also  included. 


Cost 

MDA  has  not  yet  established  baselines  for  total  costs  or  unit  costs,  both 
fundamental  markers  most  programs  use  to  measure  progress.  Consequently, 
for  the  sixth  year,  GAO  has  not  been  able  to  assess  MDA’s  actual  costs  against 
a  baseline  of  either  total  costs  or  unit  costs.  MDA  planned  to  establish  such 
baselines  in  2008  in  response  to  past  GAO  recommendations,  but  has  delayed 
this  until  2009.  GAO  was  able  to  assess  the  cost  performance  on  individual 
contracts,  and  project  an  overrun  at  completion  of  between  $2  billion  and  $3 
billion.  However,  because  in  some  cases  the  budgeted  costs  at  completion — 
the  basis  for  our  projection — has  changed  significantly  over  time  as 
adjustments  were  made,  this  projection  does  not  capture  as  cost  growth  the 
difference  between  the  original  and  current  budgeted  costs  at  completion.  In 
one  case,  these  costs  increased  by  approximately  five  times  its  original  value. 

Performance  and  Testing 

While  MDA  completed  several  key  tests  that  demonstrated  enhanced 
performance  of  the  BMDS,  all  elements  of  the  system  had  test  delays  and 
shortfalls.  Overall,  testing  achieved  less  than  planned.  For  example,  none  of 
the  six  Director’s  test  knowledge  points  established  by  MDA  for  2008  were 
achieved.  Poor  performing  target  missiles  have  been  a  persistent  problem. 
Testing  shortfalls  have  slowed  the  validation  of  models  and  simulations, 
which  are  needed  to  assess  the  system’s  overall  performance.  Consequently, 
the  performance  of  the  BMDS  as  a  whole  can  not  yet  be  determined. 


What  GAO  Recommends 


GAO  recommends  that  the  MDEB 
assess  how  the  transparency  and 
accountability  of  MDA  acquisitions 
can  be  strengthened  without  losing 
the  benefits  of  MDA’s  existing 
flexibilities.  Meanwhile,  MDA 
should  improve  its  cost  and  test 
baselines;  tie  modeling  and 
simulation  needs  into  test 
objectives;  provide  more  time  to 
analyze  tests;  better  coordinate 
with  independent  testers; 
synchronize  development, 
manufacturing,  and  fielding  with 
testing  and  validation;  complete  a 
key  developmental  test;  and 
strengthen  the  basis  for  capability 
declarations.  DOD  agreed  with  10 
of  the  11  recommendations  and 
partially  agreed  with  one. 

To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  GAO-09-338. 

For  more  information,  contact  Paul  Francis  at 
(202)  512-4841  orfrancisp@gao.gov. 


Schedule 

Although  fewer  tests  have  been  conducted  than  planned,  the  production  and 
fielding  of  assets  has  proceeded  closer  to  schedule.  Except  for  no  ground- 
based  interceptors  being  delivered,  all  other  radars,  standard  missiles,  and 
software  were  delivered  as  planned.  However,  some  deliveries,  such  as 
enhanced  Exoatmospheric  Kill  Vehicles,  will  now  precede  test  results.  In 
most  cases,  MDA  has  also  reduced  the  bases  it  planned  to  use  to  declare  when 
capabilities  are  operational  in  the  field.  Thus,  fielding  decisions  are  being 
made  with  a  reduced  understanding  of  system  effectiveness. 

Transparency,  Accountability,  and  Oversight 

Improvement  in  this  area  has  been  limited.  The  Missile  Defense  Executive 
Board  (MDEB)  has  acted  with  increased  authority  in  providing  oversight  of 
MDA  and  the  BMDS.  However,  transparency  and  accountability  into  MDA’s 
work  is  limited  by  the  management  fluidity  afforded  through  the  lack  of  cost 
baselines,  an  unstable  test  baseline,  continued  use  of  development  funds  to 
produce  assets  for  fielding,  and  renewed  potential  for  transferring  work  from 
one  predefined  block  to  another.  A  better  balance  must  still  be  struck 
between  the  information  Congress  and  the  Department  of  Defense  need  to 
conduct  oversight  of  the  BMDS  and  the  flexibility  MDA  needs  to  manage 
across  the  portfolio  of  assets  that  collectively  constitute  the  system’s 
capability.  At  this  point,  the  balance  does  not  provide  sufficient  information 
for  effective  oversight. 
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GAO 

^^Accountability  *  Integrity  *  Reliability 

United  States  Government  Accountability  Office 
Washington,  DC  20548 


March  13,  2009 
Congressional  Committees 

The  Missile  Defense  Agency  (MDA)  has  spent  almost  $56  billion  since  its 
initiation  in  2002  on  developing  and  fielding  a  Ballistic  Missile  Defense 
System  (BMDS)  and  is  on  course  to  spend  about  $50  billion  more  over  the 
next  5  years.  In  2002,  the  President  directed  the  Department  of  Defense 
(DOD)  to  “deploy  a  set  of  initial  missile  defense  capabilities  beginning  in 
2004”. 1  MDA  began  delivering  an  initial  capability  in  late  2004,  as  directed, 
and  deployed  an  initial  capability  in  2005  by  concurrently  developing  and 
fielding  BMDS  assets.2  Though  this  approach  facilitated  the  rapid 
deployment  of  an  initial  BMDS  capability,  as  MDA  has  proceeded  beyond 
that  initial  capability,  it  has  been  less  successful  in  fostering  adequate 
knowledge  of  system  capabilities  prior  to  manufacturing  and  fielding 
BMDS  assets. 

In  its  fiscal  year  2002,  2007,  and  2008  National  Defense  Authorization  Acts, 
Congress  directed  GAO  to  assess  the  cost,  schedule,  testing,  and 
performance  progress  that  MDA  is  making  in  developing  the  BMDS.3  We 


1  This  initial  BMDS  capability  was  to  defend  the  U.S.  homeland,  deployed  troops,  friends, 
and  allies  against  ballistic  missiles  of  all  ranges  in  all  phases  of  flight.  MDA  was  tasked  with 
carrying  out  the  President’s  direction. 

2  According  to  MDA,  the  agency  was  expected  to  field  an  initial  increment  of  missile 
defense  capability  that  provides  initial  protection  of  the  entire  United  States  from  North 
Korea,  partial  protection  of  the  United  States  from  the  Middle  East  threat  and  protection  of 
deployed  forces,  allies,  and  friends  with  terminal  defenses.  MDA  fielded  a  limited  capability 
that  included  initial  versions  of  Ground-based  Midcourse  Defense;  Aegis  Ballistic  Missile 
Defense;  Patriot  Advanced  Capability-3;  and  Command,  Control,  Battle  Management,  and 
Communications  elements.  MDA  expected  to  enhance  these  initial  capabilities  and  over 
time,  produce  an  overarching  BMDS  capable  of  protecting  the  United  States,  deployed 
forces,  friends,  and  allies  from  ballistic  missile  attacks  of  all  ranges. 

3  National  Defense  Authorization  Act  for  Fiscal  Year  2002,  Pub.  L.  No.  107-107,  §  232(g) 
(2001);  John  Warner  National  Defense  Authorization  Act  for  Fiscal  Year  2007,  Pub.  L.  No. 
109-364,  §  224  (2006);  and  National  Defense  Authorization  Act  for  Fiscal  Year  2008,  Pub.  L. 
No.  110-181,  §  225  (2007). 
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have  delivered  assessments  covering  fiscal  years  2003  through  2007.4  This 
report  assesses  the  progress  made  during  fiscal  year  2008  toward  BMDS 
goals  as  well  as  the  progress  MDA  made  in  improving  accountability  and 
transparency  through  its  agency  operations,  management,  processes,  and 
new  block  strategy.  This  report  also  includes  an  appendix  that  addresses 
the  Senate  Armed  Services  Committee’s  request  that  we  review  the 
reasons  behind  and  the  effects  on  BMDS  development  of  the  cancellation 
of  a  Ground-based  Midcourse  Defense  flight  test  designated  FTG-04. 

To  assess  progress  during  fiscal  year  2008,  we  examined  the 
accomplishments  of  10  BMDS  elements  that  MDA  is  developing  and 
fielding:  the  Aegis  Ballistic  Missile  Defense  (Aegis  BMD);  Airborne  Laser 
(ABL);  BMDS  Sensors;  Command,  Control,  Battle  Management,  and 
Communications  (C2BMC);  Ground-based  Midcourse  Defense  (GMD); 
Kinetic  Energy  Interceptors  (KEI);  Multiple  Kill  Vehicles  (MKV);  Space 
Tracking  and  Surveillance  System  (STSS);  Targets  and  Countermeasures; 
and  Terminal  High  Altitude  Area  Defense  (THAAD).5  These  elements 
collectively  account  for  about  80  percent  of  MDA’s  research  and 
development  budget.  We  also  examined  MDA’s  Fiscal  Year  2008  Statement 
of  Goals,  Program  Execution  Reviews,  test  plans  and  reports,  production 
plans,  and  Contract  Performance  Reports.  We  interviewed  officials  within 
program  offices  and  within  MDA  functional  directorates,  such  as  the 
Directorate  for  Cost  Estimating.  In  addition,  we  discussed  each  element’s 
test  program  and  its  results  with  the  BMDS  Operational  Test  Agency  and 
DOD’s  Office  of  the  Director,  Operational  Test  and  Evaluation  (DOT&E). 


4  We  did  not  assess  MDA’s  progress  in  fiscal  year  2002  as  the  agency  did  not  establish  goals 
for  that  fiscal  year.  We  delivered  the  following  reports  for  fiscal  years  2003  through  2007: 
GAO,  Missile  Defense:  Actions  Are  Needed  to  Enhance  Testing  and  Accountability, 
GAO-04-409  (Washington,  D.C.:  Apr.  23,  2004);  Defense  Acquisitions:  Status  of  Ballistic 
Missile  Defense  Program  in  2004,  GAO-05-243  (Washington,  D.C.:  Mar.  31,  2005);  Defense 
Acquisitions:  Missile  Defense  Agency  Fields  Initial  Capability  but  Falls  Short  of 
Original  Goals,  GAO-06-327  (Washington,  D.C.:  Mar.15,  2006 f,  Defense  Acquisitions: 
Missile  Defense  Acquisition  Strategy  Generates  Results  blit  Delivers  Less  at  a  Higher 
Cost,  GAO-07-387  (Washington,  D.C. :  Mar.15,  2007);  and  Defense  Acquisitions:  Progress 
Made  in  Fielding  Missile  Defense,  but  Progjum.  Is  Short  of  Meeting  Goals,  GAO-08-448 
(Washington,  D.C.:  Mar.  14,  2008). 

5  The  BMDS  also  includes  an  11th  element,  Patriot  Advanced  Capability-3,  which  has  been 
transferred  to  the  Army  for  production,  operation,  and  sustainment.  This  report  does  not 
evaluate  Patriot  Advanced  Capability-3  because  its  initial  development  is  complete  and  is 
now  being  managed  by  the  Army. 
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In  assessing  progress  made  toward  improving  accountability  and 
transparency,  we  held  discussions  with  officials  in  MDA’s  Directorate  of 
Business  Operations  to  determine  whether  its  new  block  structure 
improved  accountability  and  transparency  of  the  BMDS.  In  addition,  we 
reviewed  pertinent  sections  of  the  U.S.  Code  to  compare  MDA’s  current 
level  of  accountability  with  federal  acquisition  laws.  We  also  interviewed 
officials  from  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  to  discuss  the  oversight  role  of  the  Missile 
Defense  Executive  Board.  Additionally,  we  reviewed  the  board’s  charter  to 
determine  its  oversight  responsibility.  Our  scope  and  methodology  is 
discussed  in  more  detail  in  appendix  V. 

We  conducted  this  performance  audit  from  May  2008  to  March  2009  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives. 


Background 


MDA’s  mission  is  to  develop  an  integrated  and  layered  system  to  defend 
the  United  States  and  its  deployed  forces,  friends,  and  allies  against 
ballistic  missile  attacks.  The  BMDS  aims  to  engage  all  ranges  of  ballistic 
missiles  during  all  phases  of  flight.  This  challenging  expectation  requires 
complex  coordination  within  an  integrated  system  of  defensive 
components — space-based  sensors,  surveillance  and  tracking  radars, 
advanced  interceptors,  and  a  battle  management,  command,  control,  and 
communication  component. 

A  typical  engagement  scenario  to  defend  against  an  intercontinental 
ballistic  missile  would  occur  as  follows: 

•  Infrared  sensors  aboard  early-warning  satellites  detect  the  hot  plume 
of  a  missile  launch  and  alert  the  command  authority  of  a  possible 
attack. 

•  Upon  receiving  the  alert,  land-  or  sea-based  radars  are  directed  to 
track  the  various  objects  released  from  the  missile  and,  if  so  designed, 
to  identify  the  warhead  from  among  spent  rocket  motors,  decoys,  and 
debris. 

•  When  the  trajectory  of  the  missile’s  warhead  has  been  adequately 
established,  an  interceptor — consisting  of  a  kill  vehicle  mounted  atop  a 
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booster — is  launched  to  engage  the  threat.  The  interceptor  boosts 
itself  toward  a  predicted  intercept  point  and  releases  the  kill  vehicle. 

•  The  kill  vehicle  uses  its  onboard  sensors  and  divert  thrusters  to  detect, 
identify,  and  steer  itself  into  the  warhead.  With  a  combined  closing 
speed  of  up  to  10  kilometers  per  second  (22,000  miles  per  hour),  the 
warhead  is  destroyed  above  the  atmosphere  through  a  “hit  to  kill” 
collision  with  the  kill  vehicle. 

•  Some  interceptors  use  sensors  to  steer  themselves  into  the  inbound 
ballistic  missile.  Inside  the  atmosphere,  these  systems  kill  the  ballistic 
missile  using  a  range  of  mechanisms  such  as  direct  collision  between 
the  missile  and  the  inbound  ballistic  missile  or  killing  it  with  the 
combined  effects  of  a  blast  fragmentation  warhead  (heat,  pressure, 
and  grains/shrapnel)  in  cases  where  a  direct  hit  does  not  occur. 

Table  1  provides  a  brief  description  of  10  BMDS  elements  currently  under 

development  by  MDA. 


Table  1:  MDA  BMDS  Elements 

BMDS  element 

Missile  defense  role 

Aegis  Ballistic  Missile  Defense 

Aegis  BMD  is  a  ship-based  missile  defense  system  designed  to  destroy  short-  to 
intermediate-range  ballistic  missiles  during  the  midcourse  phase  of  their  flight;  its 
capability  is  being  expanded  to  include  the  terminal  phase  of  flight.  Aegis  BMD's  mission 
is  twofold— an  engagement  capability  against  regional  ballistic  missile  threats  providing 
the  BMDS  with  its  first  mobile,  global,  deployable  and  proven  capability  that  can  destroy 
ballistic  missiles  both  above  and  within  the  atmosphere,  as  well  as  a  forward-deployed 
combatant  to  search,  detect,  and  track  ballistic  missiles  of  all  ranges  and  transmit  track 
data  to  the  BMDS,  performing  a  strategic  role  in  homeland  defense.  To  date,  18  ships 
have  been  upgraded  for  the  Aegis  BMD  mission.  MDA  is  planning  to  procure  147  Aegis 
BMD  missiles— the  Standard  Missile-3  (S M -3) — from  calendar  years  2004  through  2013. 

Airborne  Laser 

ABL  is  an  air-based  missile  defense  system  designed  to  destroy  all  classes  of  ballistic 
missiles  during  the  boost  phase  of  their  flight.  ABL  employs  a  high-energy  chemical  laser 
to  rupture  a  missile's  motor  casing,  causing  the  missile  to  lose  thrust  or  flight  control. 

MDA  plans  to  demonstrate  proof  of  concept  in  a  system  demonstration  in  2009. 

BMDS  Sensors 

MDA  is  developing  radars  for  fielding  as  part  of  the  BMDS.  The  BMDS  uses  these 
sensors  to  identify  and  track  ballistic  missiles.  The  ultimate  goal  is  to  provide  continuous 
tracking  of  ballistic  missiles  in  all  phases  of  flight  and  increase  the  probability  for 
successful  intercept. 

Command,  Control,  Battle  Management 
and  Communications 

C2BMC  is  the  integrating  element ofthe  BMDS.  Its  role  is  to  provide  deliberate  planning, 
situational  awareness,  sensor  management,  and  battle  managementfor  the  integrated 
BMDS. 
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BMDS  element 

Missile  defense  role 

Ground-based  Midcourse  Defense 

G  M  D  is  a  ground-based  missile  defense  system  designed  to  destroy  intercontinental 
ballistic  missiles  during  the  midcourse  phase  of  their  flight.  Its  mission  is  to  protect  the 

U.S.  homeland  against  ballistic  missile  attacks  from  North  Korea  and  the  Middle  East. 
Currently,  GMD  has  fielded  24  interceptors  with  the  original  configuration,  known  as 
Capability  Enhancement-1  (CE-I).  GMD  has  recently  begun  emplacing  a  new 
configuration  of  the  kill  vehicle  known  as  the  Capability  Enhancement-ll  (CE-II).  This 
configuration  was  designed  to  replace  obsolete  parts.  MDA  is  planning  on  fielding  44 
interceptors  at  Fort  G  reely,  Alaska,  and  Vandenberg  Air  Force  Base,  California,  by  fiscal 
year  2011.  MDA  also  plans  to  field  10  interceptors  in  Europe. 

Kinetic  Energy  Interceptors 

KEI  is  a  mobile  land-based  missile  defense  system  designed  to  destroy  medium, 
intermediate,  and  intercontinental  ballistic  missiles  during  the  boost  and  midcourse 
phases  of  their  flight.  The  agency  plans  to  conduct  the  first  booster  flight  test  in  2009. 

The  KEI  capability  could  be  expanded  to  sea  basing  in  subsequent  blocks. 

Multiple  Kill  Vehicle 

The  M  KV  is  being  designed  as  a  spiral  improvement  to  counter  advancements  in  the 
threatfor  midcourse  interceptors.  This  approach  mitigates  the  need  to  pinpoint  a  single 
lethal  object  in  a  threat  cluster  by  using  numerous  kill  vehicles  to  engage  all  objects  that 
might  be  lethal.  The  system  under  development  consists  of  a  carrier  vehicle  housing  a 
number  of  smaller  kill  vehicles,  which  would  primarily  benefit  the  Ground-based  and 

Kinetic  Energy  interceptors  as  well  as  the  Aegis  BMD  SM-3.  To  mitigate  risk,  MDA  has 
initiated  a  parallel  acquisition  with  a  second  contractor.  Because  MKV  is  in  the 
technology  development  stage,  it  does  not  project  an  initial  capability  date,  but  the 
program  expects  that  the  capability  could  be  available  by  2017. 

Space  Tracking  and  Surveillance  System 

STSS  is  designed  to  be  a  low-orbit  constellation  of  space-based  sensors  that  is  able  to 
observe  targets  in  all  phases  of  trajectory.  MDA  intends  to  launch  two  demonstration 
satellites  in  2009.  If  the  demonstration  satellites  perform  successfully,  MDA  plans  to 
have  an  operational  capability  of  next-generation  satellites. 

Targets  and  Countermeasures 

MDA  maintains  a  series  of  targets  used  in  BMDS  flight  tests  to  present  authentic  threat 
scenarios.  The  targets  are  designed  to  encompass  the  full  spectrum  of  threat  missile 
ranges  and  capabilities.  In  2005,  MDA  began  developing  a  new  family  of  targets,  the 
Flexible  TargetFamily  (FTF),  which  was  to  represent  evolving  threats  of  all  ranges. 
However,  in  2008,  MDA  narrowed  the  FTF  focus  to  developing  one  long-range  72-inch 
target,  the  LV-2.  The  first  launch  of  this  target  is  scheduled  for  2009. 

Terminal  High  Altitude  Area  Defense 

THAAD  is  a  ground-based  missile  defense  system  designed  to  destroy  short-  and 
medium-range  ballistic  missiles  during  the  terminal  phase  of  flight,  both  inside  and 
outside  ofthe  atmosphere.  Its  mission  is  to  defend  deployed  U.S.  forces  and  population 
centers.  MDA  plans  to  field  a  fire  unit,  which  includes  24  missiles,  in  2010  and  a  second 
unit  in  2011.  MDA  also  plans  to  field  two  additional  fire  units,  which  includes  24  missiles 
each,  in  2012  and  2013,  respectively. 

Source:  MDA  data. 


To  manage  BMDS  development,  MDA  uses  an  acquisition  strategy  defined 
by  a  block  structure.  From  its  inception  in  2002  through  2007,  MDA 
developed  BMDS  capability  in  biennial  increments,  ultimately  delivering 
two  blocks — Block  2004  and  Block  2006.  These  2-year  blocks  each  built  on 
preceding  blocks  and  enhanced  the  development  and  capability  of  the 
BMDS.  However,  in  response  to  recommendations  from  GAO,  in 
December  2007  MDA  announced  a  new  block  structure  that  was  intended 
to  improve  the  program’s  transparency,  accountability,  and  oversight.  The 
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new  blocks  are  not  based  on  biennial  time  periods,  but  instead  focus  on 
fielding  capabilities  that  address  particular  threats.  Because  the  new  block 
structure  is  not  aligned  to  regular  time  periods,  multiple  blocks  are 
underway  concurrently.  Table  2  details  the  current  blocks  and  categories 
included  in  the  BMDS  block  structure. 

Table  2:  MDA  Block  Construct 

Block 

Description 

Block  1.0:  Defend  U.S.  from  Limited  North 
Korean  Long-Range  Threats 

Provides  an  initial  capability  to  protect  the  United  States  from  a  limited  North  Korean 
attack.  It  is  the  most  mature  capability  and  will  be  the  first  block  delivered  to  the 
warfighter. 

Block  2.0:  Defend  Allies  and  Deployed 
Forces  from  Short-  to  Medium-Range 
Threats  in  One  Region/Theater 

Includes  capabilities  needed  to  defend  allies  and  deployed  forces  from  short-  to 
medium-range  threats  in  one  region/theater. 

Block  3.0:  Expand  Defense  of  the  U.S.  to 
Include  Limited  Iranian  Long-Range 
Threats3 

Builds  on  the  foundation  established  in  Block  1.0  and  includes  capabilities  needed  to 
expand  the  defense  of  the  United  States  against  limited  Iranian  long-range  threats. 

Block  4.0:  Defend  Allies  and  Deployed 
Forces  in  Europe  from  Limited  Iranian  Long- 
Range  Threats 

Builds  on  the  foundation  established  by  Blocks  1.0  and  3.0  capabilities  needed  to 
-  defend  allies  and  deployed  forces  in  Europe  from  limited  Iranian  long-range  threats  and 
to  expand  protection  of  the  U.S.  homeland. 

Block  5.0:  Expand  Defense  of  Allies  and 
Deployed  Forces  from  Short-  to 
Intermediate-Range  Threats  in  Two 
Regions/Theaters 

Builds  on  the  foundation  established  by  Block  2.0  and  includes  capabilities  needed  to 
expand  defense  of  allies  and  deployed  forces  from  short-  to  intermediate-range  threats 
in  two  regions/theaters. 

Categories 

Capability  Development 

Includes  BMDS  elements  and  other  development  elements  thatare  notbaselined  in  the 
existing  agency  block  structure,  such  as  ABL,  KEI,  and  MKV.  These  programs  have 
knowledge  points  tailored  to  critical  risks. 

Sustainment 

Funding  for  Contractor  Logistics  Support  and  other  operation  and  support  activities. 

Mission  Area  Investment 

Investments  that  cut  across  several  blocks  and  cannot  be  reasonably  allocated  to  a 
specific  block.  Examples  include  modeling  and  simulation  and  intelligence  and  security. 

MDA  Operations 

Contains  operations  support  functions  such  as  MDA  headquarters  management. 

Source:  MDA  data. 


“Block  3.0  is  subdivided  into  three  sections:  3.1,  3.2,  and  3.3.  Block  3.0  will  focus  on  more 
sophisticated  sensors  and  algorithms,  and  therefore  includes  upgrades  to  the  Ground-based 
Interceptors,  sensors,  and  the  C2BMC  system  to  allow  discrimination  of  the  threat  missile.  MDA  is 
pursuing  two  parallel  and  complementary  approaches  to  counter  complex  countermeasures.  The  full 
implementation  ofthis  approach  will  be  conducted  in  phases,  with  the  firstphase  referred  to  as  Near 
T erm  Discrimination  (Block  3. 1/3. 2)  and  the  second  phase  as  Improved  Discrimination  and  System 
T rack.  (Block  3.3). 
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MDA  uses  a  Statement  of  Baselines  and  Goals  to  report  modifications  to 
established  block  baselines.6  For  those  blocks  that  are  currently  or  will 
soon  be  underway,  block  baselines  are  created  to  make  a  firm 
commitment  to  Congress.  The  Statement  of  Goals  also  includes  the 
following: 

•  BMDS  Baseline  Capabilities  -  Assets  and  engagement  sequence 
groups  that  will  be  made  available  for  fielding  for  a  particular  block.7 
During  2008,  cost  baselines  were  under  development  for  Block  2.0  and 
Block  increments  3. 1/3.2.  MDA  established  schedule  and  performance 
baselines  for  Blocks  1.0,  2.0,  3.1,  and  3.2  in  2008. 

•  BMDS  Capability  Goals  -  Assets  and  engagement  sequence  groups 
expected  to  be  made  available  for  future  blocks. 

•  Adversary  Benchmarks  -  Adversary  missile  systems  used  for  block 
performance  estimates. 

•  BMDS  Budget  Breakdowns  -  Detailed  fielding,  development,  and 
integration  budgets  for  each  block  and  BMDS  Capability  Development 
activity. 

MDA  also  uses  an  incremental  declaration  process  to  designate  BMDS 
capability  for  its  blocks.  Three  capability  designations  are  applied  to  all 
BMDS  elements,  their  hardware  and  software  components,  and 
engagement  sequence  groups.  This  allows  these  BMDS  features  to  play  a 
limited  role  in  system  operations  before  they  have  attained  their  expected 
level  of  capability.  Each  capability  designation  in  the  delivery  schedule 
represents  upgraded  capacity  to  support  the  overall  function  of  BMDS  in 
its  mission  as  well  as  the  level  of  MDA  confidence  in  the  system’s 
performance.  The  designations  are  defined  as  follows: 


6  Beginning  in  2009,  this  report — previously  referred  to  as  the  Statement  of  Goals — will  be 
called  the  BMDS  Accountability  Report. 

'  BMDS  hardware  and  software  are  grouped  into  engagement  sequence  groups,  each  of 
which  is  the  specific  combination  of  all  sensors,  weapons,  and  C2BMC  capability  that  are 
needed  to  detect,  track,  and  intercept  a  particular  threat.  The  engagement  sequence  group 
construct  was  created  as  an  engineering  tool  to  provide  a  simple  representation  of  BMDS 
capabilities,  integration,  and  functionality  and  is  defined  as  a  unique  combination  of  detect- 
control-engage  functions  performed  by  BMDS  subsystems  used  to  engage  a  threat  ballistic 
missile. 
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•  Early  Capability  Delivery  signifies  readiness  for  contingency  use.  At 
this  point,  MDA  has  determined  that  the  capability  can  be  utilized  by 
the  BMDS.  When  integrated,  the  capability  must  be  adequately 
demonstrated  to  build  sufficient  confidence  that  it  will  safely  perform 
as  intended  without  degrading  the  existing  capabilities  of  the  BMDS. 

•  Partial  Capability  Delivery  is  an  intermediate  state  of  maturity 
indicating  that  a  capability  has  been  shown  through  testing  to  perform 
as  intended  in  certain  scenarios.  At  this  point,  MDA  is  sufficiently 
confident  that  the  capability  can  support  the  warfighter’s  partially 
mission-capable  objectives  and  logistics  support  is  adequate  to  achieve 
defensive  operations. 

•  Full  Capability  Delivery  is  the  point  at  which  a  capability  satisfies  the 
BMDS  block  objectives  and  is  considered  to  be  completely  mature  and 
ready  for  full  operational  use. 

MDA’s  capability  goals  for  fiscal  year  2008  for  Blocks  1.0,  2.0,  3.1,  and  3.2 

are  shown  in  table  3. 
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Table  3:  Fiscal  Year  2008  Capability  Goals  for  Blocks  1.0,  2.0,  and  3.1/3.2 

Engagement  Sequence  Group 

2008  Planned 
Capability  Deliveries 

Block  1.0 — Initial  Defense  of  U.S.  from  North  Korea  Expected  Completion:  Fiscal  Year  2009 

Ground-Based  Interceptor  (GBI)  Launch  on  COBRA  DANE/Upgraded  Early  Warning  Radar 

Early 

GBI  Engage  on  COBRA  DANE/Upgraded  Early  Warning  Radar 

Full 

GBI  Engage  on  forward-based  AN/TPY-2  radar 

Full 

GBI  Engage  on  Sea-based  X-band  radar 

Early 

GBI  Launch  on  Sea-based  X-band  radar 

Early 

Block  2.0 — Initial  Defense  of  Allied  Forces  Expected  Completion:  Fiscal  Year  2011 

SM-2  Engage  on  shipboard  Aegis  radar  (AN/SPY-1) 

Early 

SM-3  Engage  on  shipboard  Aegis  radar  (AN/SPY-1) 

Full 

SM-3  Launch  on  remote  on  shipboard  Aegis  radar  (AN/SPY-1) 

Early 

THAAD  Interceptor  Engage  on  AN/TPY-2  radar  in  the  terminal  mode 

Early 

Block  3.1/3.2 — Initial  Defense  of  U.S.  from  Iran  Expected  Completion:  Fiscal  Year  2013 

GBI  Engage  on  COBRA  DANE/Upgraded  Early  Warning  Radar  Mod  1  (Fylingdales,  UK; 

Forward-Based  mobile  radar  (AN/TPY-2)) 

Partial 

GBI  Launch  on  shipboard  Aegis  radar  Mod  1  (Fylingdales,  UK;  Sea-based  X-band  radar) 

Early 

Source:  GAO  analysis  of  MDA  data. 

Note:  In  addition  to  the  engagement  sequence  groups  listed  above,  as  of  October  2007  MDA  had 
planned  to  declare  the  capability  of  several  more  engagementsequence  groups  in  fiscal  year2008. 
Howeverthese  were  excluded  from  the  February  2008  Statementof  Goals.  MDA  continues  to  work 
toward  declaring  these  additional  engagementsequence  groups. 


Cost  Tracking 
Deficiencies  Hinder 
Assessment  of  Cost 
Performance 


MDA  has  not  yet  established  baselines  for  total  costs  or  unit  costs,  both 
fundamental  markers  that  most  programs  use  to  measure  progress.  MDA 
had  planned  to  establish  total  cost  baselines  at  the  element  and  block 
levels  in  2008,  but  the  initial  set  of  total  cost  baselines  will  not  be  available 
until  the  spring  of  2009.  Similarly,  MDA  has  not  established  unit  costs  for 
selected  assets,  such  as  GBIs.  Consequently,  for  the  sixth  year,  we  have 
been  unable  to  assess  MDA’s  overall  progress  on  total  or  unit  cost.  While 
MDA  plans  to  establish  some  total  cost  baselines  in  2009,  most  efforts  will 
not  be  captured  in  a  baseline.  MDA  also  plans  to  establish  unit  costs, 
another  improvement,  but  is  considering  a  narrower  definition  of  unit  cost 
than  is  used  by  other  weapon  system  programs.  MDA’s  definition  will 
report  a  subset  of  procurement  costs  called  flyaway  costs,  which  only 
includes  the  major  piece  of  equipment  and  excludes  all  research  and 
development  as  well  as  some  procurement  costs — those  for  support 
equipment  and  spares.  Moreover,  these  unit  costs  will  only  be  tracked 
within  those  blocks  that  are  baselined,  which  will  represent  a  minority  of 
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those  assets  being  produced  and  fielded.  Without  total  cost  baselines  in 
place,  the  BMDS  concept  continually  evolves,  as  indicated  by  the  number 
of  realignments  to  the  program  of  work  at  the  individual  contract  level. 
While  the  changing  nature  of  the  BMDS  and  the  lack  of  total  cost  baselines 
precludes  analysis  of  total  cost  progress,  we  were  able  to  analyze 
contractor  fiscal  year  performance  on  the  current  work  under  the 
contract.  We  were  also  able  to  project  overruns  or  underruns  at 
completion  for  BMDS  contracts  using  the  contracts’  current  budgeted 
costs  at  completion  as  a  basis  for  our  projections.  However,  in  some  cases, 
the  current  budgeted  cost  at  completion  changed  significantly  over  time. 

In  one  case,  the  budgeted  cost  at  completion  increased  by  approximately 
five  times  its  original  value.  Our  analysis  of  fiscal  year  2008  progress 
shows  that  several  prime  contractors  exceeded  budgeted  costs. 


Absence  Of  Cost  Baselines  To  provide  accountability,  major  defense  acquisition  programs  are 
Prevents  Assessment  Of  required  by  statute  to  document  program  goals  in  an  acquisition  program 
System-Level  Costs  baseline.8  MDA  is  not  yet  required  to  establish  an  acquisition  program 

baseline  because  of  the  acquisition  flexibilities  it  has  been  granted. 
However,  Congress  has  enacted  legislation  requiring  MDA  to  establish 
some  baselines.9  Baselines  serve  an  important  discipline  both  by  ensuring 
that  the  full  cost  commitment  is  considered  before  embarking  on  major 
development  efforts  and  by  identifying  cost  growth  as  a  program 
proceeds.  Since  we  began  annual  reporting  on  missile  defense  in  2004,  we 
have  been  unable  to  assess  overall  progress  on  cost — that  is,  comparing 
BMDS  baselined  costs  with  actual  costs.  For  example,  under  the  prior 
block  structure,  we  reported  that  BMDS  costs  grew  by  at  least  $1  billion, 


10  U.S.C.  §  2435  requires  an  approved  program  baseline  for  major  defense  acquisition 
programs.  The  BMDS  program  meets  the  definition  of  a  major  defense  acquisition  program, 
which  is  defined  in  10  U.S.C.  §  2430;  however,  the  requirement  to  establish  a  baseline  is  not 
triggered  until  entry  into  system  development  and  demonstration.  Because  the  BMDS  has 
not  yet  formally  entered  the  acquisition  cycle,  it  has  not  yet  been  required  to  meet  the 
minimum  requirements  of  section  2435. 

9  The  Fiscal  Year  2005  National  Defense  Authorization  Act,  Pub.  L.  No.  108-375  §  234(e), 
required  the  Director,  MDA,  to  establish  and  report  annually  to  Congress  a  cost,  schedule, 
and  performance  baseline  for  each  block  configuration  being  fielded.  Modification  to  the 
baseline  and  variations  against  the  baseline  must  also  be  reported.  In  addition,  the  National 
Defense  Authorization  Act  for  Fiscal  Year  2008,  Pub.  L.  No.  110-181  §  223(g)  required  that 
no  later  than  the  submittal  of  the  budget  for  fiscal  year  2009,  MDA  shall  “establish 
acquisition  cost,  schedule  and  performance  baselines”  for  BMDS  elements  that  have 
entered  the  equivalent  of  system  development  and  demonstration  or  are  being  produced 
and  acquired  for  operation  fielding. 
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but  the  total  cost  growth  could  not  be  determined  because  MDA  did  not 
account  for  all  costs  for  a  given  block. 

In  response  to  recommendations  we  made  in  March  2008,  MDA  agreed  to 
develop  cost  estimates  and  to  provide  independent  verification  for  blocks 
outlined  under  its  new  approach.  Upon  conclusion  of  those  estimates, 
MDA  will  develop  cost  baselines.  In  addition,  on  April  1,  2008,  the 
Director,  MDA,  testified  before  a  Senate  Armed  Services  Subcommittee 
that  cost  baselines  for  the  new  block  structure  would  be  available  by  the 
end  of  2008.  As  of  January  2009,  the  agency  had  not  yet  developed  full  cost 
baselines  for  any  blocks.  MDA  plans  to  have  these  cost  baselines  for 
Blocks  2.0,  3.1  and  3.2  completed,  independently  reviewed  by  DOD’s  Cost 
Analysis  Improvement  Group,  and  released  by  the  spring  of  2009.  The  only 
information  that  was  available  for  this  report  was  limited  to  budget 
projections  for  BMDS  blocks  and  capability  development  for  fiscal  years 
2008  through  2013,  totaling  approximately  $42  billion. 

Even  with  the  release  of  some  block  cost  estimates  in  2009,  all  block  costs 
will  not  be  baselined  and  no  date  has  been  established  for  when  the 
remaining  block  costs  will  be  baselined.  MDA  does  not  plan  to  baseline 
Block  1.0  costs,  but  will  provide  the  actual  costs  of  the  block  since  it  is 
near  completion.  Full  cost  baselines  for  Block  2.0  are  anticipated  to  be 
available  in  the  spring  of  2009,  but  only  portions  of  Block  3.0  and  none  of 
Block  4.0  and  5.0  costs  will  be  baselined  at  this  time.  As  figure  1  shows,  if 
MDA  does  complete  baselines  as  planned,  they  will  only  cover  about  26 
percent  of  its  block  and  capability  development  costs. 
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Figure  1 :  Estimated  Percentage  of  Total  BMDS  Block  and  Capability  Development 
Funds  through  Fiscal  Year  2013  Expected  to  Be  Baselined  in  2009 
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Source:  GAO  analysis  of  MDA’s  Fiscal  Year  2009  Budget  Estimate  Submission  and  J  anuary  2008  Statement  of  Goals. 

Note:  Analysis  is  based  on  MDA's  fiscal  year  2009  projected  funding  through  fiscal  year  2013  from 
the  February  2008  request.  Funding  includes  defense-wide  resources  projected  for  MDA. 

At  this  point,  MDA  plans  to  baseline  between  2  and  26  percent  of  BMDS 
block  and  capability  development  costs  from  fiscal  years  2010  to  2013  as 
depicted  above.  MDA  has  not  determined  when  other  blocks  will  be 
baselined.  If  other  blocks  were  to  be  baselined  before  2013,  the  percentage 
of  funding  baselined  would  be  increased.  The  rapid  decline  in  percentage 
of  baselined  funds  also  shows  that  initial  baselines  are  being  set  late  in  a 
block’s  duration.  For  example,  Block  2.0  will  be  completed  within  2  years 
of  its  baseline  being  set.  If  baselines  are  to  facilitate  management  and 
oversight,  they  will  have  to  be  set  sooner  for  Block  3.3  and  beyond. 
Additionally,  agency  officials  stated  that  although  cost  estimates  will  be 
developed  for  individual  capability  development  efforts — such  as  ABL, 

KEI,  and  MKV — the  agency  does  not  plan  to  baseline  their  costs  until  these 
elements  are  matured  and  moved  into  a  defined  block.  MDA  may 
eventually  baseline  these  elements  as  part  of  a  block  once  a  firm 
commitment  can  be  made  to  Congress.  The  budgets  for  capability 
development  elements  account  for  approximately  $22  billon — or  more 
than  half  of  MDA’s  fiscal  year  2008  6-year  Future  Years  Defense  Plan  for 
BMDS  blocks  and  capability  development. 
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Planned  Unit  Cost 
Reporting  Will  Not  Be 
Comprehensive 


Major  defense  acquisition  programs  are  required  by  statute  to  report 
certain  unit  costs  to  Congress,10  track  unit  cost  growth  against  their 
original  and  current  baseline  estimates,  and  perform  an  additional 
assessment  of  the  program  if  certain  cost  growth  thresholds  are  reached.11 
This  cost  monitoring  mechanism  helps  ensure  that  programs  are  being 
held  accountable.  MDA  is  not  yet  required  to  report  these  unit  costs 
because  of  the  acquisition  flexibilities  it  has  been  granted  by  DOD,  but 
Congress  has  enacted  legislation  requiring  MDA  to  provide  unit  cost 
reporting  data  for  certain  BMDS  elements,  and  MDA  does  plan  to  develop 
and  report  unit  costs  for  some  of  its  assets  in  the  spring  of  2009. 12  The 
agency  has  also  established  thresholds  for  reporting  cost  growth. 
However,  the  approach  MDA  is  taking,  while  an  improvement,  provides  a 
much  less  comprehensive  assessment  of  unit  cost  compared  to  the 
traditional  acquisition  costs  that  are  typically  reported  for  major  defense 
acquisition  programs.  Normally,  unit  costs  are  reported  in  two  ways:  (1) 
program  acquisition  unit  cost,  which  is  the  total  cost  for  the  development 
and  procurement  of,  and  system-specific  military  construction  for,  the 
acquisition  program  divided  by  the  number  of  fully  configured  end  items 
to  be  produced,  or  (2)  average  procurement  unit  cost,  which  is  the  total  of 
all  funds  programmed  to  be  available  for  obligation  for  procurement 
divided  by  the  number  of  end  items  to  be  procured.13 

MDA’s  development  of  the  BMDS  outside  of  DOD’s  normal  acquisition 
process  makes  it  difficult  to  compare  the  actual  unit  cost  of  a  delivered 
asset  with  its  planned  unit  cost.  For  example,  MDA  plans  to  only  report 


10  U.S.C.  §  2432  requires  DOD  to  submit  to  Congress  a  Selected  Acquisition  Report  (SAR) 
for  each  major  defense  acquisition  program  that  includes  the  program  acquisition  unit  cost 
for  each  program.  Unless  waived  by  the  Secretary  of  Defense,  this  requirement  applies 
when  funds  have  been  appropriated  for  the  program  and  the  program  has  proceeded  to 
system  development  and  demonstration.  Development  programs  that  have  not  entered 
system  development  and  demonstration  may  submit  a  limited  SAR.  MDA  submits  a  limited 
SAR  that  does  not  include  program  acquisition  unit  costs. 

11  10  U.S.C.  §  2433  (commonly  referred  to  as  Nunn-McCurdy). 

12  National  Defense  Authorization  Act  for  Fiscal  Year  2008,  Pub.  L.  No.  110-181  §  223(g) 
requires  MDA  to  provide  unit  cost  reporting  data  for  each  BMDS  element  that  has  entered 
the  equivalent  of  the  systems  development  and  demonstration  phase  of  acquisition  or  is 
being  produced  and  acquired  for  operational  use  and  secure  independent  estimation  and 
verification  of  such  cost  reporting  data.  How  MDA  was  to  calculate  these  unit  costs  was 
not  specified. 

13  10  U.S.C.  §  2432. 
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recurring  unit  flyaway  costs  for  the  blocks  that  are  baselined.14  Figure  2 
reveals  the  significant  reduction  in  standard  areas  of  costs  covered  by 
MDA’s  approach  compared  to  that  normally  reported  for  major  defense 
acquisition  programs. 


Figure  2:  Difference  in  Traditional  Unit  Cost  Reporting  and  MDA’s  Unit  Cost  Reporting 
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Procurement 
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Flyaway  cost 


Procurement 

•Prime  equipment 


Source:  GAO  analysis. 


MDA’s  decision  to  report  only  flyaway  unit  costs  will  not  capture  research 
and  development  costs  associated  with  BMDS  assets — which  account  for 
more  than  97  percent  of  the  nearly  $56  billion  MDA  costs  to  date.  In 
addition,  the  procurement  costs  for  initial  spares  and  support  equipment 
are  not  included.  Thus,  while  the  flyaway  cost  baseline  will  provide 
visibility  into  changes  in  recurring  manufacturing  costs,  it  will  not  provide 


14  Flyaway  cost  refers  to  the  cost  of  procuring  prime  mission  equipment  (e.g.,  an  aircraft, 
ship,  tank,  etc.).  It  is  funded  with  Procurement  appropriations  and  is  part  of  the  Investment 
cost  category.  This  term  includes  the  Work  Breakdown  Structure  elements  of  Prime 
Mission  Equipment,  System  Engineering/Program  Management,  System  Test  and 
Evaluation,  Warranties,  and  Engineering  Changes. 
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a  basis  for  comparison  with  the  costs  of  other  DOD  programs.  If  a  cost 
increase  occurs  in  research  and  development  or  nonrecurring 
procurement,  it  will  not  be  reported  in  MDA’s  unit  cost. 

Agency  officials  told  us  that  the  reason  for  using  flyaway  unit  costs  was 
the  new  MDA  block  structure.  They  further  explained  that  within  the 
block  structure,  there  are  many  cases  where  MDA  procures  and  delivers  a 
weapon  system  for  more  than  one  block  and,  in  some  cases,  the  same 
configuration  in  more  than  one  block.  For  example,  THAAD  deliveries  are 
included  in  Blocks  2.0  and  5.0.  Agency  officials  cite  this  as  a  key  difference 
between  MDA  and  other  defense  programs.  For  MDA,  most  of  the 
development  costs  are  assigned  to  the  first  block  where  a  capability  is 
delivered  and  very  little  of  the  development  cost  is  assigned  to  subsequent 
blocks.  MDA  officials  further  stated  that  if  the  agency  were  to  use  the 
standard  unit  cost  methodology,  it  would  show  very  dissimilar  unit  costs 
between  the  first  and  subsequent  block  deliveries  and  the  difference  could 
not  be  explained  by  learning  curves  and  manufacturing  efficiencies.  MDA 
officials  also  told  us  that  they  chose  unit  flyaway  cost  for  unit  cost 
reporting  because  flyaway  cost  provides  a  better  measure  of  what  the 
individual  system  components  cost  to  procure.  However,  MDA  is  not 
precluded  from  also  determining  and  reporting  unit  costs  by  taking  the 
entire  cost  of  the  asset  being  developed  without  regard  to  the  capability  or 
block  for  which  it  is  originally  developed. 


Frequent  Realignments  Without  a  firm  cost  commitment  or  baseline  in  place  for  the  full  scope  of 

Indicate  That  Full  Scope  Is  the  BMDS,  in  some  cases  the  work  currently  under  contract  changes 

Not  Yet  Determined  frequently.  These  changes  manifest  themselves  in  realignments  that  often 

add  scope,  cost  and  time  to  the  value  of  the  work  under  the  contract. 
Since  contracts  began  on  the  BMDS,  MDA  has  performed  31  realignments 
where,  in  some  cases,  the  period  of  stability  between  these  major 
realignments  average  only  between  1  to  2  years.  These  frequent  changes 
indicate  that  the  total  BMDS  effort  has  not  been  fully  determined  and  is 
likely  to  grow.  While  we  have  been  able  to  make  an  assessment  of 
contractor  costs,  that  assessment  is  limited  to  the  current  approved 
program. 
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Until  total  and  unit  cost  baselines  are  established,  the  only  tool  for  us  to 
use  in  assessing  BMDS  costs  is  the  costs  reported  on  individual  contracts 
under  BMDS’s  Earned  Value  Management  System.15  All  BMDS  contracts 
that  we  assessed  have  a  cost  and  schedule  baseline  against  which 
progress,  measured  by  cost  and  schedule  performance,  can  be  measured. 

It  is  appropriate  for  a  program  to  realign  its  current  status  with  the 
remaining  contractual  effort  when  officials  conclude  that  the  baseline  no 
longer  provides  valid  performance  assessment  information.16  A  program 
can  realign  its  current  status  with  the  remaining  contractual  effort  through 
rebaselines,  replans,  and  restructures. 

•  A  rebaseline  is  a  more  general  term  to  describe  a  major  realignment  of 
the  performance  measurement  baseline  used  to  better  correlate  the 
work  plan  with  the  baseline  budget,  scope,  and  schedule  and  can  refer 
to  replans  and  restructures  as  well. 

•  A  replan  is  a  reallocation  of  schedule  or  budget  for  the  remaining 
effort  within  the  existing  constraints  of  the  contract. 

•  A  restructure  includes  adding  funds  to  the  performance  management 
budget  that  exceed  the  value  of  the  negotiated  contract  and  result  in  a 
contract  modification. 

For  purposes  of  this  report,  we  refer  to  each  of  these  types  of  program 
changes  as  realignments.  Since  work  under  the  BMDS  contracts  first 
began,  there  have  been  a  total  of  31  realignments  to  the  program  which 
have  added  nearly  $14  billion  dollars  to  the  value  of  the  work  under  the 
contracts.  Table  4  shows  the  BMDS  elements’  realignments  since  contract 
start. 


15  Earned  Value  Management  is  a  program  management  tool  that  integrates  the  technical, 
cost,  and  schedule  parameters  of  a  contract.  During  the  planning  phase,  an  integrated 
baseline  is  developed  by  time  phasing  budget  resources  for  defined  work.  As  work  is 
performed  and  measured  against  the  baseline,  the  corresponding  budget  value  is  “earned.” 

16  According  to  the  Over  Target  Baseline  and  Over  Target  Schedule  Handbook,  May  7, 
2003. 
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Table  4:  Analysis  of  Contractor  Realignments  from  Contract  Start  through  Fiscal 
Year  2008 


Contract  start 

Date  of  the  last 
realignments9 

Total  number 
realignments 

ABL 

Nov-96 

J  une-07 

6 

Aegis  BMD 

Oct-03 

Mar-06 

1 

C2BMC 

Feb-02 

Nov-06 

1 

GMDb 

J  an-01 

Ongoing  as-of  Sep-08 

8 

KEI 

Dec-03 

Apr-08 

4 

MKV 

J  an-04 

J  uly-07 

4 

Sensors 

Apr-03 

N/A 

0 

STSS 

Apr-02 

Oct-07 

1 

THAAD 

Aug-00 

May-08 

5 

Targets  and 
Countermeasures 

Dec-03 

J  une-08 

1 

Total 

31 
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Total  contract  value 
increase  due  to 
realignments 

Total  period  of 
performance  increase 
due  to  realignments 
(in  months) 

Average  time 
between  realignments 

Average  contract  value 
increase  per  year 
due  to  realignments 

Averageperiod 
of  performance 
increase  per  year 
due  to  realignments 
(in  months) 

$2,590,881,491 

80 

2  years 

$218,947,732 

7 

0 

0 

4  years  11  months 

0 

0 

36,514,950 

0 

6  years  7  months 

5,546,575 

0 

8,086,607,706 

27 

1  year 

1,054,774,918 

4 

1,639,800,000 

20 

1  year  2  months 

345,000,000c 

4 

51,213,935 

14 

1  year  2  months 

10,974,415 

3 

0 

0 

N/A 

0 

0 

232,293,329 

13 

6  years  5  months 

36,201,558 

2 

1,179,000,000 

15 

1  year  7  months 

146,000,000c 

2 

41,300,000 

0 

4  years  9  months 

8,694,737 

0 

$13,857,611,411 

Source:  GAO  analysis  of  MDA  data. 

"Dates  for  some  elements  reflectwhen  realignments  were  completed  and  not  necessarily  when 
realignments  were  incorporated  into  costand  schedule  performance  reporting. 

“The  GMD  program  began  a  restructure  during  the  fiscal  year  that  includes  a  proposal  to  add 
between  $350  million  and  $580  million  to  the  contract  value  as  well  as  39  months  to  the  period  of 
performance.  S  ince  the  replan  is  still  ongoing  and  has  not  yet  been  placed  on  contract,  this 
information  is  nottotaled  in  the  table  above. 

‘The  realignment  data  provided  for  the  KEI  and  THAAD  programs  included  rounding  for  contract 
value  increases;  therefore  we  have  rounded  the  average  contract  value  increase  per  year  since 
contract  start  to  reflect  this. 


Some  programs  realigned  more  often  than  others.  For  example,  GMD 
realigned  work  under  its  contract  every  year  on  average  since  its  contract 
start  in  2001,  adding  nearly  4  months  and  close  to  $1.1  billion  to  the  time 
and  value  of  the  work  under  the  contract  with  each  realignment.  KEI 
realigned  its  contract  about  every  14  months  on  average,  adding  more  than 
$345  million  and  4  months  every  year.  Since  contract  start  in  1996,  ABL 
also  added  more  than  $218  million  to  the  value  of  the  work  under  its 
contract  every  year  on  average.  Additionally,  ABL  adds  approximately  7 
months  to  its  period  of  performance  every  year  on  average — more  than 
any  other  element. 
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During  fiscal  year  2008,  5  of  10  BMDS  elements  performed  a  realignment — 
KEI,  Targets  and  Countermeasures,  GMD,  THAAD,  and  STSS.  The  KEI 
replan  in  April  2008  reflected  an  8-month  delay  to  the  booster  flight  test 
date  because  of  technical  issues  experienced  by  the  program  over  the  past 
2  years.  Since  the  replan,  the  booster  flight  test  has  been  further  delayed  to 
the  fourth  quarter  of  fiscal  year  2009.  However,  during  the  replan,  the 
program  did  not  extend  the  period  of  performance  or  add  value  to  the 
work  under  the  contract.  In  June  2008,  a  delivery  order  under  the  Targets 
and  Countermeasures  element  that  is  developing  a  new  family  of  targets — 
the  FTF — performed  a  rebaseline  adding  more  than  $41  million  to  the 
value  of  the  work  under  the  contract  but  not  extending  the  period  of 
performance.  The  program  changed  major  milestone  delivery  dates  as  a 
result  of  manufacturing  delays  for  some  systems,  caused  principally  by 
qualification  program  failures,  subsequent  redesigns,  and  requalification 
efforts. 

GMD,  THAAD,  and  STSS  added  time  and  money  to  the  value  of  the  work 
under  their  contracts  during  the  fiscal  year.17  GMD’s  ongoing  restructure 
includes  a  proposal  to  add  between  $350  million  and  $580  million  to  the 
value  of  the  work  under  contract  and  more  than  3  years  to  the  period  of 
performance.  This  ongoing  restructure  rephases  and  rescopes  ongoing 
efforts  to  refine  European  capability  requirements  and  to  adjust  program 
content  as  well  as  perform  weapon  system  integration,  perform  flight  test 
planning,  and  work  to  develop  the  two-stage  booster  among  other  tasks. 
During  its  realignment  in  May  2008,  THAAD  added  approximately  $80 
million  and  3  months  citing  cost  effects  from  insufficient  target 
availability.  In  October  2007,  STSS  replanned  work  citing  funding 
constraints  and  the  addition  of  STSS  software  upgrades.  This  resulted  in 
the  program  changing  its  launch  date  from  December  2007  to  July  2008 
and  adding  approximately  $232  million  to  the  value  of  the  work  under 
contract  and  13  months  to  its  period  of  performance.  Since  the  replan,  the 
program  has  further  delayed  launch  of  its  demonstrator  satellite  to  the 
third  quarter  of  fiscal  year  2009. 


17  The  GMD  program’s  replan  began  in  fiscal  year  2008  but  is  currently  ongoing  and,  as 
such,  was  not  totaled  into  table  4. 
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MDA  Contractors  Overran 
Fiscal  Year  Cost  and 
Schedule 


Our  analysis  of  contractor  costs  indicates  that  during  fiscal  year  2008, 

MDA  contractors  collectively  overran  budgeted  costs  by  $152.4  million.18 
These  overruns  occurred  in  11  of  14  MDA  contracts  we  reviewed,  with  the 
STSS  contract  accounting  for  more  than  50  percent  of  the  total.19  Based  on 
cost  performance  during  the  fiscal  year  and  using  formulas  accepted 
within  the  cost  community,  we  estimate  that  at  completion  the  cumulative 
overrun  in  the  contractors’  budgeted  costs  could  be  from  about  $2.0  billion 
to  $3.0  billion.  Our  projections  are  based  on  the  current  budgeted  costs  at 
completion  for  each  contract  we  assessed,  which  represents  the  total 
current  planned  value  of  the  contract.20  However,  the  budgeted  costs  at 
completion,  in  some  cases,  have  grown  significantly  over  time.  For 
example,  the  ABL  contractor  reported  budgeted  costs  at  completion 
totaling  about  $724  million  in  1997,  but  as  depicted  in  table  5,  that  cost  has 
since  grown  to  about  $3.6  billion.  Our  assessment  only  reveals  the  overrun 
or  underrun  since  the  latest  adjustment  to  the  budget  at  completion.  It 
does  not  capture,  as  cost  growth,  the  difference  between  the  original  and 
current  budgeted  costs  at  completion.  As  a  result,  comparing  the 
underruns  or  overruns  for  MDA  programs  in  table  5  with  cost  growth  on 
major  defense  acquisition  programs  is  not  appropriate  because  those 
major  defense  acquisition  programs  have  established  their  full  scope  of 
work  as  well  as  developed  total  cost  baselines,  while  these  have  not  been 
developed  for  MDA  programs.  Our  analysis  is  presented  in  table  5. 
Appendix  II  provides  further  details  on  the  cost  and  schedule  performance 
outlined  in  the  table. 


18  To  determine  if  contractors  are  executing  the  work  planned  within  the  funds  and  time 
budgeted,  each  BMDS  program  office  requires  its  prime  contractor  to  provide  monthly 
reports  detailing  cost  and  schedule  performance. 

19  We  analyzed  three  task  orders  for  the  MKV  program  issued  as  part  of  an  Indefinite 
Delivery/Indefinite  Quantity  contract  as  well  as  three  contracts  managed  by  the  Aegis  BMD 
element.  We  assessed  one  contract  for  each  of  the  other  eight  elements.  Indefinite 
Delivery/Indefinite  Quantity  contracts  provide  for  an  indefinite  quantity,  within  stated 
limits,  of  supplies  or  services  during  a  fixed  period. 

20  The  current  budgeted  costs  at  completion  are  as-of  September  30,  2008. 
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Table  5:  Prime  Contractor  Fiscal  Year  2008  and  Cumulative  Cost  and  Schedule 

Performance 

(Dollars  in  millions) 

Fiscal  year  2008 

Fiscal  year  2008 

BMDS  contract 

cost  performance8 

schedule  performance8 

ABL 

(10.6) 

2.2 

Aegis  BMD  Weapon  System 

(7.0) 

(5.1) 

Aegis  BMD  SM-3  CLIN  9  (20  Block 

1A  missiles)” 

(3.9) 

3.9 

Aegis  BMD  SM-3  CLIN  1  (27  Block 

1A  missiles)” 

3.0 

(7.6) 

C2BMC 

(9.8) 

(3.6) 

GMD 

53.9 

(77.4) 

KEId 

(8.3) 

(8.5) 

MKV  Task  Order  6  (Prototype 

Carrier  Vehicle  Seeker)8 

(1.4) 

(1.5) 

MKV  Task  Order  7  (Engagement 
Management  Algorithms)8 

1.4 

0.0 

MKV  Task  Order  8  (Hover Test 

Bed)8 

(10.7) 

(0.0) 

Sensors 

(2.2) 

(27.4) 

STSSf 

(87.9) 

1.9 

Targets  and  Countermeasures 

(35.7) 

23.2 

THAAD9 

(33.5) 

(7.4) 

Total 

(152.4) 

(107.4) 
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Cumulative 

cost 

performance 

Cumulative 

schedule 

performance 

Percentage 
of  contract 
completed 

Estimated  contract 
overrun/underrun 
at  completion 

Budget  at 
completion 

Period  of  performance 

(84.8) 

(23.6) 

91.1 

Overrun  of  $89.7  to  $95.4 

$3,626.7 

Nov.  1996  -  Feb.  2010 

0.0 

(8.4) 

81.1 

Overrun  of  $1.9  to  $12.2 

1,247.0 

Oct.  2003  -  Sept.  2010 

2.3 

(0.1) 

94.2 

Underrun  of  $7.5 

179.0 

Aug. 2006  -  Aug.  2008 

3.3 

(7.0) 

46.3 

Underrun  of  $6.6  to  overrun  of 
$0.7 

237.5 

May  2007  -  Apr.  2010 

(24.3) 

(7.1) 

71.1 

Overrun  of  $37.1  to  $76.8 

1,040.0 

J  an.  2002  -  Dec.  2009 

(1,027.9) 

(130.3) 

84.0 

Overrun  of  $950.2  to  $1,251.3 

14,934.9 

J  an.  2001  -  Dec  2011 

(2.6) 

(21.3) 

13.9 

N/A 

6,068.3 

Dec.  2003  -  Oct2014 

(1.1) 

(0.6) 

78.3 

Overrun  of  $1.6  to  $2.5 

19.3 

Nov.  2006  -  May  2009 

1.7 

0.1 

52.8 

Underrun  of  $3.9  to  $3.2 

43.9 

Dec.  2006  -  May  2010 

(10.3) 

0.3 

81.4 

Overrun  of  $5.7  to  $13.8 

48.0 

Dec.  2006  - J  an  2009 

22.0 

(9.6) 

80.7 

Underrun  of  $25.0  to  overrun  of 
$9.1 

1,125.2 

Mar.  2003  -  Dec  2010 

(319.3) 

(17.8) 

53.2 

Overrun  of  $621.7  to  $1,157.9 

1,603.0 

Apr  2002  -  Sept.  2011 

(52.8) 

(6.4) 

84.5 

Overrun  of  $63.7  to  $75.9 

1,056.4 

Dec.  2003  -  Dec  2009 

(228.7) 

(16.5) 

91.4 

Overrun  of  $252.0  to  $274.0 

4,649.4 

Aug.  2000  -  Sept.  2009 

(1,722.5) 

(248.3) 

Overrun  of  $1,980.8  to  $2,959.0 

Source: 

Contract  Performance  Reports  (data);  GAO  (analysis). 

Note:  Comparing  the  percentage  of  total  overrun  to  total  budget  at  completion  for  MDA  contracts  with 
percentage  of  total  cost  growth  for  major  acquisition  defense  programs  that  are  past  milestone  B  is 
not  appropriate  because  the  major  defense  acquisition  programs  have  established  their  full  scope  of 
work  as  well  as  developed  total  cost  baselines,  while  these  have  not  been  developed  for  MDA 
programs. 

“Costperformance  here  is  defined  as  the  difference  between  the  budgetforthe  work  performed  and 
the  actual  cost  of  work  performed;  while  the  schedule  performance  is  the  difference  between  the 
budgeted  cost  of  planned  work  and  the  budgeted  cost  of  work  performed.  Negative  cost  performance 
(budget  overruns)  and  negative  schedule  performance  (less  work  performed  than  planned)  are  shown 
with  parentheses  around  the  dollar  amounts. 

“The  Aegis  BMD  SM-3  contractor  began  work  on  contract  line  item  number  (CLIN)  9  in  February  2007 
that  concluded  in  August2008  forthe  acquisition  of  an  additional  20  SM-3  Block  1A  missiles.  All 
corresponding  analysis  is  based  on  data  through  August  2008. 

'The  Aegis  BMD  SM-3  contractor  began  reporting  performance  on  CLIN  1  in  August2007.  This  CLIN 
is  forthe  production  of  a  fourth  lotof  27  Block  1A  missiles. 

dWe  could  not  estimate  the  likely  outcome  of  the  KEI  contract  at  completion  because  a  trend  cannot 
be  predicted  until  15  percentof  the  planned  work  is  complete. 
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"Out  of  the  five  task  orders  open  during  fiscal  year  2008,  there  was  sufficient  cost  performance  data 
to  report  on  the  three  listed  above. 

'The  STSS  contract  includes  line  items  forwork  that  do  not  necessarily  apply  to  the  program  being 
launched  in  the  third  quarter  of  fiscal  year  2009.  Removing  these  line  items  from  our  analysis,  the 
program's  contract  would  be  considered  78%  complete. 

’Earned  Value  data  for  the  THAAD  contract  is  reported  undertwo  CLINs,  1  and  10.  We  report  only 
the  contractor's  cost  and  schedule  performance  for  CLIN  1  because  it  represents  the  majority  of  the 
total  work  performed  under  the  contract.  CLIN  10  provides  for  Patriot  Common  Launcher  initiatives 
funded  by  the  Army's  LowerTier  Program  Office. 

Technical  difficulties  caused  most  elements  to  overrun  their  fiscal  year 
2008  budgeted  costs.  For  example,  STSS  attributed  most  of  its  overrun  of 
approximately  $87.9  million  to  hardware  problems  on  the  program’s 
second  space  vehicle  including  the  flight  communication  box  and  the  main 
spacecraft  computer.  The  box  overheated  during  testing  which  required  a 
thorough  test  of  the  unit.  Upon  successful  completion  of  this  testing,  it 
was  determined  it  did  not  require  a  replacement.  In  addition,  the  program 
had  a  failure  in  the  main  spacecraft  computer  for  which  the  program  office 
initially  recommended  the  removal  of  the  entire  computer  from  the 
spacecraft.  However,  after  extensive  research  and  testing,  the  program 
manager  determined  that  the  event  with  the  spacecraft  was  an  unverifiable 
failure  with  a  low  probability  of  occurrence  and  low  mission  impact  and 
decided  not  to  remove  the  computer  from  the  spacecraft  to  resolve  the 
issue.  However,  as  a  result  of  these  issues,  the  launch  was  delayed  from 
April  2008  to  at  least  the  third  quarter  of  fiscal  year  2009. 

The  MKV  Task  Order  6  and  ABL  contractors  also  experienced  technical 
difficulties.  The  MKV  contractor  for  Task  Order  6  reported  cost  overruns 
during  the  fiscal  year  of  $1.4  million  due  mostly  to  software  development 
issues  and  late  delivery  of  government-furnished  components.  ABL’s  fiscal 
year  cost  overruns  of  $10.6  million  were  mainly  related  to  late  deliveries  of 
key  laser  system  components  and  the  acquisition  or  refurbishment  of  the 
Beam  Control/Fire  Control  system  components.  For  example,  a  major 
component  of  the  laser  system  required  redesign  and  fabrication,  delaying 
planned  delivery  and  installation  onto  the  aircraft.  Also,  multiple  Beam 
Control/Fire  Control  hardware  components  either  were  not  refurbished  to 
specification  or  failed  initial  testing,  delaying  delivery  and  integration 
testing.  The  overall  effect  was  an  approximate  1-month  slip  that  the 
contractor  believes  will  be  made  up  in  time  to  make  the  current  lethality 
demonstration  planned  for  the  end  of  fiscal  year  2009. 

Three  elements’  contracts — Aegis  BMD’s  contract  for  27  SM-3  Block  1A 
missiles,  MKV  Task  Order  7,  and  GMD — performed  below  their  fiscal  year 
budgeted  costs  by  nearly  $68.3  million  with  the  GMD  element  accounting 
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for  approximately  $53.9  million  of  that.  The  GMD  element’s  underruns 
occurred  partially  because  the  contractors  delayed  or  eliminated  some 
planned  work.  For  example,  the  GMD  program  did  not  emplace  the  three 
GBIs  it  expected  to  in  fiscal  year  2008  or  conduct  either  of  its  two  planned 
flight  tests  as  scheduled  during  the  fiscal  year.  As  a  result,  it  employed  less 
labor  than  originally  intended.  Drivers  for  the  MKV  Task  Order  7  contract’s 
fiscal  year  cost  underruns  include  a  restructuring  of  the  effort  and 
decisions  to  use  one  rather  than  several  approaches  for  coordinated 
attack,  and  using  less  manpower  than  originally  planned  in  its 
procurement  and  software  efforts.  Lastly,  the  Aegis  BMD  contract  for  27 
SM-3  Block  1A  missiles  underran  its  fiscal  year  2008  budget  by 
approximately  $3  million  due  in  part  to  spending  less  than  planned  for 
engineering  efforts  with  the  missile’s  third  stage  component  as  well  as 
adjustments  made  in  program  management,  labor  efficiencies,  and 
material  transfers  in  the  missile’s  fourth  stage  component. 


While  Some  Tests 
Succeeded,  Others 
Were  Deferred; 
Overall  System 
Performance  Cannot 
Yet  Be  Assessed 


Although  several  tests  showed  progress  in  individual  elements  and  some 
system-level  capabilities,  all  BMDS  elements  experienced  test  delays  and 
shortfalls,  in  part  due  to  problems  with  the  availability  and  performance  of 
target  missiles.  Most  significantly,  GMD  was  unable  to  conduct  either  of  its 
planned  intercept  attempts  during  fiscal  year  2008,  however  it  was  able  to 
conduct  one  delayed  intercept  test  in  December  2008.  As  a  result,  key 
performance  capabilities  of  the  current  configuration  of  the  GMD  kill 
vehicles  may  not  be  demonstrated  and  the  new  configuration  is  being 
fielded  prior  to  flight  testing.  As  a  consequence  of  testing  problems,  none 
of  the  six  MDA  Director’s  test  knowledge  points  for  2008  were  achieved. 
Poor  performance  of  targets  continues  to  be  a  problem  that  caused  several 
tests  to  either  fail  in  part  or  in  whole.  Shortfalls  in  testing  have  delayed 
validating  the  models  and  simulations  that  are  used  to  assess  the  overall 
performance  of  the  BMDS  as  a  whole.  Consequently,  comprehensive 
assessments  of  the  capabilities  and  limitations  of  the  BMDS  are  not 
currently  possible  and  therefore  MDA  still  does  not  have  the  capability  to 
model  or  simulate  BMDS  capability  from  enemy  missile  launch  to  its 
engagement. 


Test,  Targets,  and 
Performance  Challenges 
Continued  during  Fiscal 
Year  2008  for  Several 
Elements 


During  fiscal  year  2008,  all  BMDS  elements  experienced  delays  in 
conducting  tests,  most  were  unable  to  accomplish  all  objectives,  and 
performance  challenges  continued  for  many.  Moreover,  the  inability  of 
MDA  to  conduct  its  full  fiscal  year  2008  flight  test  campaign  as  planned 
precluded  the  agency  from  collecting  key  information  specified  by  the 
Director,  MDA — known  as  Director’s  test  knowledge  points — to  make 
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certain  decisions  at  critical  points  in  some  BMDS  programs.  Table  6  below 
summarizes  test  results  and  target  performance  for  BMDS  elements  during 
the  fiscal  year. 


Table  6:  Test  and  Targets  Issues 

Element 

Tests/activities 
conducted  as 
scheduled 

All  objectives 
achieved 

Target  issues 

ABL 

No 

Yes 

N/A 

Aegis  BMD 

No 

No 

Target  availability 
delayed  key  test 
from  2008  until  at 
least  the  third 
quarter  fiscal  year 
2009. 

C2BMC 

No 

No 

N/A 

GMD 

No 

No 

Target  failed  to 
release 

countermeasures 
during  December 

2008  flight 
test— FTG-05.3 

KEI 

No 

No 

N/A 

MKV 

No 

Nob 

N/A 

Sensors 

No 

No 

Target  failed  to 
release 

countermeasures 
during  J  uly  2008 
testing  (FTX-03). 

STSS 

No 

No 

N/A 

Targets  and 
Countermeasures 

No 

No 

FTF  delivery 
delayed  and 
experienced  cost 
growth. 

THAAD 

No 

No 

Target  experienced 
anomaly  during  a 
September  flight  test 
resulting  in  a  no-test. 

Sources:  GAO  (presentation);  MDA  (data). 


"This  flight  test  was  originally  scheduled  for  fiscal  year  2008,  but  was  later  executed  in  fiscal  year 
2009. 

“The  MKV  program  was  able  to  achieve  its  objective  in  the  first  quarter  of  fiscal  year  2009. 
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As  a  result  of  test  delays,  MDA  restructured  its  flight  test  plan  for  fiscal 
year  2009,  increasing  the  number  of  tests  and  compressing  the  amount  of 
time  to  analyze  and  prepare  for  subsequent  tests.  For  example,  MDA  plans 
to  conduct  14  of  18  flight  tests  in  the  third  and  fourth  quarter  of  fiscal  year 
2009.  MDA’s  past  performance  raises  questions  about  whether  this  is 
realistic.  In  fiscal  year  2008,  MDA  had  planned  to  conduct  18  flight  tests, 
but  it  only  accomplished  10,  plus  it  had  several  flight  tests  delayed  into 
2009  from  2008.  An  MDA  official  acknowledged  that  the  2009  plan  is 
aggressive,  but  stated  that  it  can  be  achieved.  Specifics  of  each  element’s 
testing  experience  during  fiscal  year  2008  follow. 

According  to  Aegis  BMD  officials,  budgetary  constraints  prompted  the 
Aegis  BMD  element  to  delay  some  tests,  reducing  the  number  of  tests 
planned  for  2008.  However,  the  program  was  able  to  successfully  complete 
its  first  test  involving  two  non-separating  targets,  conduct  a  short-range 
ballistic  missile  intercept,  and  participate  in  a  THAAD  intercept  during  the 
fiscal  year.  The  program  also  planned  to  participate  in  a  BMDS-level 
ground  test  during  the  year,  but  the  test  was  delayed  until  at  least  the 
second  quarter  of  fiscal  year  2009  because  of  real-world  events.  Finally, 
Aegis  BMD  standard  missile  flight  tests  showed  that  interoperability  issues 
persist  between  THAAD  and  Aegis  BMDS  with  respect  to  correlation  and 
object  reporting. 

ABL  experienced  delays  during  fiscal  year  2008,  but  achieved  all  of  its 
primary  test  objectives.  The  program  planned  to  complete  the  installation 
of  its  high  energy  laser  on  the  aircraft  by  June  2008  in  preparation  for 
testing.  However,  it  was  not  completed  until  August  2008  because  of 
problems  with  activating  some  of  the  laser’s  subsystems.  The  program 
delayed  the  final  testing  of  the  laser  until  the  problems  could  be  resolved. 
Once  the  problems  were  resolved,  the  ABL  program  was  able  to  complete 
functionality  testing  of  the  laser  in  September  2008. 

C2BMC  experienced  delays  in  conducting  tests,  but  achieved  several  test 
objectives.  For  example,  software  upgrade  verification  testing  slipped 
from  fiscal  year  2008  to  2009  but  the  program  was  able  to  participate  in 
many  other  system-level  ground  and  flight  tests  during  the  year  that 
enabled  the  program  to  demonstrate  multiple  capabilities,  including 
situational  awareness  and  sensor  management.21  The  C2BMC  element 


21  Situational  awareness  is  defined  as  the  degree  to  which  the  perception  of  the  current 
environment  mirrors  reality. 
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extended  development  for  its  next  software  release,  6.4,  by  more  than  a 
year  because  of  delays  in  system-level  BMDS  testing  and  challenges  in 
developing  the  network  server  for  version  6.2,  as  well  as  unplanned  work 
to  incorporate  effects  from  earth  rotation  in  the  6.4  C2BMC  planning 
architecture.  C2BMC  added  earth  rotation  effects  to  address  a  requirement 
that  Spiral  6.2  and  later  releases  have  the  ability  to  model  the  true  extent 
of  ranges  for  long-range  threats.  Finally,  C2BMC  is  still  developing  its 
capability  to  generate  a  single  track  from  multiple  sensors  through  a  new 
resource  management  function,  the  Global  Engagement  Manager.  For 
example,  the  development  team  for  this  function  had  to  modify  the  design 
of  this  function’s  new  track  processing  that  experienced  an  unacceptable 
level  of  delays  when  processing  data. 

In  fiscal  year  2008,  the  GMD  program  was  unable  to  conduct  either  of  its 
two  planned  intercept  attempts — FTG-04  and  FTG-05.  MDA  first  delayed 
and  then  later  canceled  the  FTG-04  test  in  May  2008  due  to  a  problem  with 
a  telemetry  component  in  the  interceptor’s  Exoatmospheric  Kill  Vehicle 
(EKV)  needed  to  gather  test  data.  MDA  also  delayed  FTG-05  from  fiscal 
year  2008  and  conducted  it  in  December  2008.  Over  the  past  two  years 
MDA  had  expected  to  conduct  seven  GMD  interceptor  flight  tests  by  the 
end  of  the  first  quarter  of  fiscal  year  2009.  However,  MDA  was  only  able  to 
conduct  two,  as  shown  in  figure  3. 
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Figure  3:  GMD  Reduction  in  Flight  Tests  from  January  2006  to  March  2010 


Source:  GAO  analysis  of  MDA  data. 


The  cancellation  of  FTG-04  raised  concerns  within  the  test  community  and 
members  of  Congress.  FTG-04  was  at  first  delayed  and  then  canceled. 

MDA  replaced  it  with  a  test  to  assess  sensor  capability — FTX-03.  The 
sensor  test  allowed  GMD  to  verify  fire  control  software  and  integration 
with  multiple  sensors.  The  DOT&E  was  not  consulted  on  the  decision  to 
cancel  FTG-04  and  expressed  concern  that  the  elimination  of  any  intercept 
test  reduced  the  opportunity  to  gather  data  that  might  have  increased 
confidence  in  the  models  and  simulations.  In  the  conference  report 
accompanying  the  National  Defense  Authorization  Act  for  Fiscal  Year 
2008,  conferees  expressed  concern  about  the  loss  of  the  FTG-04  flight  test 
and  requested  that  we  review  the  circumstances  and  the  effects  on  the 
BMDS.  Details  of  our  review  of  the  FTG-04  flight  test  cancellation  appear 
in  appendix  III. 
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Because  GMD  conducted  FTG-05  in  December  2008,  there  are  only  two 
full  sets  of  GMD  intercept  data  to  date  available  for  analysis  which  limits 
the  ability  to  verify  and  validate  the  models  and  simulations.22  Additionally, 
FTG-04  and  the  subsequent  test — FTG-05 — were  planned  to  present 
different  stresses  to  the  kill  vehicle  which  would  provide  critical  data 
needed  to  further  verify  the  fielded  configuration  of  the  kill  vehicle.  The 
cancellation  and  subsequent  restructuring  of  the  first  test  caused  a  delay 
in  FTG-05  from  the  third  quarter  of  fiscal  year  2008  until  December  2008. 

In  the  FTG-05  test,  the  interceptor  hit  its  intended  target.  However,  MDA 
judged  the  target  as  a  failure  because  it  failed  to  release  its 
countermeasures  as  planned.  Consequently,  all  primary  test  objectives 
were  not  achieved. 

Looking  forward  to  the  next  GMD  intercept  flight  test — FTG-06  in  at  least 
the  fourth  quarter  of  fiscal  year  2009 — MDA  is  accepting  a  higher  level  of 
risk  than  it  previously  expected  in  conducting  this  first  test  of  the  CE-II 
EKV  because  it  will  contain  several  objectives  that  had  planned  to  be 
previously  tested,  but  have  not  been.  MDA  had  set  up  an  approach  to  test 
one  new  major  component  change  at  a  time.  For  example,  MDA  had 
planned  to  test  the  CE-I  EKV  first  against  simple  targets,  then  the  CE-I 
against  a  complex  target,  and  once  that  had  been  proven  MDA  planned  to 
test  the  CE-II  EKV  against  a  complex  target.  However,  MDA  was  not  able 
to  test  the  CE-I  EKV  against  a  complex  target  due  to  a  target  failure.  Due 
to  testing  problems,  GMD  has  only  been  able  to  assess  the  CE-I  EKV  with  a 
target  without  countermeasures.  As  a  result,  the  FTG-06  flight  test  will  be 
the  first  GMD  test  assessing  both  a  CE-II  EKV  and  a  complex  target  scene. 
Adding  to  the  risk,  this  will  be  only  the  second  test  using  a  newly 
developed  FTF  LV-2  target. 

During  fiscal  year  2008,  the  KEI  program  experienced  problems  during 
testing  that  required  it  to  rework  components  which,  in  turn,  caused  a 
delay  to  subsequent  testing.  More  importantly,  due  to  technical  issues 
experienced  by  the  program  over  the  past  two  years,  the  first  booster 
flight  test — a  key  decision  point  for  the  program — has  been  delayed  by 
nearly  a  year  and  is  not  scheduled  to  occur  until  at  least  the  fourth  quarter 
of  fiscal  year  2009.  Technical  difficulties  delayed  the  MKV  program’s  fiscal 
year  2008  hover  test  until  fiscal  year  2009.  This  hover  test  will  allow  the 


22  According  to  program  officials,  an  earlier  test — FTG-02 — provided  limited  data  for 
assessment  purposes.  However,  the  data  was  incomplete  and  could  not  be  used  to  fully 
verify  and  validate  the  models  and  simulations. 
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program  to  integrate  and  test  key  components  of  the  system  in  a 
repeatable  ground-based  free  flight  environment  as  their  technologies 
reach  maturity.  Although  originally  planned  for  the  fourth  quarter  of  fiscal 
year  2008,  the  test  was  successfully  conducted  in  December  2008.  The 
STSS  program  encountered  problems  during  testing  that  forced  the 
program  to  delay  the  launch  of  its  demonstration  satellites  from  April  2008 
to  at  least  the  third  quarter  of  fiscal  year  2009.  The  program  continued  to 
experience  technical  difficulties  with  its  space  vehicles.  For  example, 
during  testing,  the  program  experienced  problems  with  its  main  spacecraft 
computer  as  well  as  an  overheating  flight  communications  box.  After 
extensive  testing,  the  program  determined  that  these  components  were 
acceptable  for  flight. 

Similarly,  in  fiscal  year  2008,  the  Sensors  element  also  experienced  flight 
test  delays  as  well  as  difficulties  in  achieving  planned  objectives  due  to 
target  performance,  but  met  some  primary  objectives.  The  element 
successfully  participated  in  other  tests  during  the  fiscal  year  which 
demonstrated  the  ability  for  the  sensors  to  acquire  and  track  a  target.  One 
test  event,  FTX-03,  provided  the  first  opportunity  for  four  key  sensors — 
Sea-based  X-band  radar,  AN/TPY-2,  Upgraded  Early  Warning  Radar,  and  an 
Aegis  BMD  radar — to  operate  in  a  more  operationally  realistic  test 
scenario.23  This  test  demonstrated  the  capability  for  the  sensors  to 
correlate  target  information  in  order  to  conduct  an  intercept  test. 

However,  the  target  failed  to  release  its  countermeasures  as  planned.  This 
failure  precluded  sensors  from  assessing  capability  against  a  dynamic 
lethal  target  scene  with  countermeasures.  As  a  result,  the  sensors  could 
not  collect  all  of  the  expected  data,  which  delayed  the  element’s  ability  to 
develop  algorithms  needed  for  the  discrimination  capability.  These 
objectives  will  need  to  be  addressed  in  future  testing.  The  BMDS 
Operational  Test  Agency  has  had  ongoing  concerns  regarding  the 
formatting,  tracking,  and  accounting  of  messages  from  GMD  sensors. 24 
The  timely  reception  of  messages  from  sensors  to  weapon  systems  is  key 
to  support  decisions  and  achieve  effective  intercepts.  Since  2000  the  BMDS 
Operational  Test  Agency  has  reported  these  concerns  to  MDA  about  poor 
data  collection  and  management  practices  involving  sensors  affecting  its 


23  AN/TPY-2  was  formerly  known  as  Forward-Based  X-Band  Transportable  radar. 

"4  The  BMDS  Operational  Test  Agency  conducts  independent  operational  assessments  of 
BMDS  capability  to  defend  the  United  States,  its  deployed  forces,  friends,  and  allies  against 
ballistic  missiles  of  all  ranges  and  in  all  phases  of  flight.  MDA  funds  all  BMDS  Operational 
Test  Agency  activities. 
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assessment  of  tests.  These  data  management  problems  prevented  the 
analysis  of  message  data,  according  to  BMDS  Operational  Test  Agency 
officials.  In  response,  the  contractor  proposed  a  message  monitoring 
system  among  communications  nodes.  Consequently,  MDA  recommended 
that  this  issue  be  closed  out,  but  the  BMDS  Operational  Test  Agency  still 
considers  the  matter  to  be  open  because  GMD  has  not  funded  the 
monitoring  system. 

THAAD  planned  to  conduct  three  intercept  attempts,  but  due  to  a  target 
failure,  it  was  only  able  to  conduct  two.  The  program  could  not  complete 
its  final  flight  test  of  the  fiscal  year  because  the  target  experienced  an 
anomaly  during  flight.  The  test  was  planned  to  be  a  BMDS-level  event  and 
was  designated  as  a  developmental  test/operational  test  mission  utilizing 
multiple  BMDS  elements  and  operationally  realistic  criteria.  The  program 
also  expected  to  demonstrate  that  it  could  launch  more  than  one  THAAD 
interceptor  during  the  engagement.  Program  officials  rescheduled  this  test 
for  the  second  quarter  of  fiscal  year  2009.  In  addition,  THAAD’s  radar  data 
collection  test,  RDC-2,  was  planned  for  2008  but  was  deleted  due  to  target 
availability  and  funding.  As  a  result,  program  officials  told  us  that  these 
test  objectives  will  be  covered  in  the  future  with  hardware-in-the-loop 
simulations  and  other  radar  events.  The  program  successfully  completed 
its  first  two  planned  tests  for  the  fiscal  year.  In  October  2007,  THAAD 
successfully  demonstrated  an  intercept  of  a  target  outside  the  earth’s 
atmosphere.  This  was  the  first  time  THAAD  had  successfully  conducted  an 
intercept  outside  of  the  atmosphere  since  1999.  Additionally,  in  June  2008, 
THAAD  completed  a  successful  intercept  of  a  separating  target.  This 
intercept  utilized  warfighter  procedures  developed  by  the  U.S.  Army  Air 
Defense  School. 


Key  MDA  Test  Knowledge  As  a  consequence  of  flight  test  delays  as  well  as  a  delay  in  a  key  ground 
Points  Not  Achieved  test,  MDA  was  unable  to  achieve  any  of  the  Director’s  test  knowledge 

points  scheduled  for  fiscal  year  2008  as  shown  in  table  7. 
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Table  7:  Status  of  Fiscal  Year  2008  Director’s  Test  Knowledge  Points 


Knowledge  point 

Knowledge  gained 

Flight  and 
ground  test 

Original 

completion 

Current 

projection 

Assess  Capability  to 

Deliver  Real-Time 
EngagementTracks 

Verification  of  initial  Global  Engagement 
Manager  capability  to  support  BMDS-level 
sensor/weapon  system  pairing. 

GTD-033 

4th  Quarter  2008 

2nd  Quarter  2009 

Verify  72-inch  Flexible 
Target  Family 

Confirmation  of  72"  performance.  Viability  of 
FTF  concept  to  efficiently  configure  and 
transport  target  to  launch  facility.  Confidence 
to  discontinue  use  of  STARS. 

FTM-15 

4th  Quarter  2008 

3rd  Quarter  2009 

Demonstrate  H  ig h- 
Acceleration  Booster 

Confirmation  of  Boost  Phase  Capability 
alternative  to  ABL  and  High  Acceleration 
Booster  for  Midcourse  Defense  (mobile  and 
fixed  sites). 

FTK-01 

4th  Quarter  2008 

4th  Quarter  2009 

Confirm  Constellation 
Affordability 

Space  sensor  performance  against 
operationally  realistic  targets  confirmed  with 
existing  Block  06  technology  (anchors 
performance-cost  baseline  forfuture  STSS). 

FTS-01 

4th  Quarter  2008 

4“'  Quarter  2009 

Verify  Capability  to 

Conduct  Launch  on 

Tactical  Digital  Information 
Link  BM  Engagement 

Assessment  of  Aegis  BMD  3.6  and  SM-3  Block 
IA  performance  and  ability  to  successfully 
engage  and  intercept  a  long-range  ballistic 
missile  target  and  to  use  an  off-board  sensor's 
track  data  via  Link-16  to  initiate  that 
engagement. 

FTM-15 

4th  Quarter  2008 

3rd  Quarter  2009 

Confirm  Constellation 
Performance 

Space  sensor  performance  against 
operationally  realistic  targets  confirmed  with 
existing  Block  06  technology  (anchors 
performance-cost  baseline  forfuture  STSS). 

FTS-03 

4th  Quarter  2008 

To  Be 

Determined 

Source:  GAO  analysis  of  MDA  data. 


aGTD-03  was  delayed  to  accommodate  a  real-world  contingency  as  requested  by  the  warfighter. 


In  May  2007,  the  Director,  MDA,  established  key  system-level  and  element- 
level  knowledge  points  to  provide  critical  information  for  making  key 
decisions  regarding  the  BMDS.  According  to  MDA,  these  knowledge  points 
are  unique  management  approaches  chosen  to  manage  MDA’s  critical 
program  risks.26  Each  knowledge  point  is  based  on  an  event  that  provides 
critical  information — or  knowledge — for  a  key  MDA  decision  requiring  the 
Director’s  approval. 


25  There  are  various  categories  of  knowledge  points.  For  example,  an  element  knowledge 
point  is  based  on  an  element  event  that  provides  critical  information  for  a  key  element 
program  decision  requiring  the  Program  Manager’s  approval.  Element  knowledge  points 
support  one  or  more  Director  knowledge  points,  and  may  be  supported  by  other 
knowledge  points. 
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In  fiscal  year  2008,  among  the  Director’s  test  knowledge  points  delayed, 
MDA  had  to  defer  the  confirmation  of  the  72”  target  performance  due  to 
delays  in  qualifying  components.  Additionally,  MDA  had  to  delay  the 
confirmation  of  the  booster  for  the  KEI  program  as  problems  were 
encountered  during  testing  of  the  nozzle. 


Poor  Target  Missile  While  targets  have  caused  problems  in  fiscal  year  2008  testing,  poor 

Performance  Continues  to  performance  of  targets  is  not  new.  Targets’  reliability  and  availability 

Hamper  BMDS  Testin0'  problems  have  significantly  affected  BMDS  development  and  testing  since 

2006,  and  issues  have  grown  even  more  problematic  in  recent  years. 
Although  target  anomalies  and  failures  have  affected  many  of  the  missile 
defense  elements,  THAAD  and  GMD  have  been  most  affected.  In  2006,  the 
THAAD  program  was  unable  to  achieve  its  first  intercept  attempt  (FTT-04) 
because  the  target  did  not  function  properly.  In  2007,  two  THAAD  radar 
characterization  tests  (RDC-lc  and  RDC-ld)  were  unsuccessful  due  to 
target  anomalies.  These  tests  flew  targets  with  characteristics  needed  for 
radar  observation  in  support  of  advanced  discrimination  algorithm 
development.  However,  target  problems  prevented  an  opportunity  for  the 
radar  to  exercise  all  of  the  planned  algorithms,  causing  a  loss  of  expected 
data.  In  addition  to  target  failure  issues,  the  THAAD  program  deferred 
some  flight  tests  because  targets  were  not  available,  which  cost  the 
program  about  $201  million.  GMD  also  experienced  similar  long-term 
effects  on  its  flight  test  schedule  when  it  was  unable  to  achieve  primary 
test  objectives  in  a  2007  intercept  attempt  (FTG-03)  due  to  a  target  failure. 

MDA’s  existing  targets  are  becoming  less  capable  of  meeting  requirements 
for  near-term  flight  tests.  These  targets  are  aging  and  likely  to  grow  even 
less  reliable  with  time;  some  components,  such  as  the  rocket  motors,  are 
more  than  40  years  old.  Among  other  things,  MDA’s  Targets  and 
Countermeasures  program  office  has  also  had  problems  incorporating 
requirements  into  contracts  and  has  experienced  problems  obtaining 
supplies  as  vendors  left  the  market  due  to  the  lack  of  business. 

To  address  the  growing  need  for  more  sophisticated  and  reliable  targets 
for  the  future  BMDS  test  program,  MDA  was  developing  a  new  family  of 
targets  called  the  FTF,  which  was  originally  intended  to  be  a  family  of  new 
short,  medium,  and  long-range  targets  with  ground-,  air-,  and  sea-  launch 
capabilities.  MDA  embarked  on  this  major  development  without 
estimating  the  cost  to  develop  the  family  of  target  missiles.  MDA 
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proceeded  to  develop  and  even  to  produce  some  FTF  targets  without  a 
sound  business  case  and,  consequently,  their  acquisition  has  not  gone  as 
planned. 26  The  funds  required  for  the  FTF  were  spent  sooner  than 
expected  and  were  insufficient  for  the  development.  Getting  the  FTF 
target’s  components  through  the  qualification  process,  however,  was  more 
difficult  and  costly  than  the  program  expected.  For  example,  MDA 
originally  planned  to  launch  the  first  FTF  target — a  72-inch  LV-2 — in  a  2008 
STSS  flight  test,  but  the  test  was  rescheduled  due  to  delays  in  satellite 
integration  and  target  affordability  and  availability.  While  many  of  the 
target  missile’s  components  are  found  on  existing  systems,  their  form,  fit, 
function,  and  the  environment  they  fly  in  have  been  changed  for  the  72- 
inch  LV-2  target.  Consequently,  many  critical  components  initially  failed 
shock  and  vibration  testing  and  other  qualification  tests  and  had  to  be 
redesigned.  The  process  was  recently  scheduled  to  be  complete  in  early 
October  2008  but,  after  several  delays,  was  not  finished  until  December 
2008.  Despite  this,  MDA  expects  the  target  to  be  complete  and  ready  for  its 
first  launch  in  a  third  quarter  fiscal  year  2009  Aegis  BMD  flight  test  (FTM- 
15). 

We  recently  reported  that  the  FTF  has  been  delayed,  costs  have  increased 
and  exceeded  $1  billion,  and  the  program’s  scope  has  been  reduced.27 
Work  on  all  but  one  of  the  FTF  target  variants,  the  72-inch  LV-2,  was 
canceled  in  June  2008,  including  plans  for  development  and  production  of 
the  second  type  of  FTF  target,  the  52-inch,  originally  scheduled  to  launch 
in  2009.  With  guidance  from  the  Missile  Defense  Executive  Board,  MDA  is 
currently  conducting  a  comprehensive  review  of  the  targets  program  to 
determine  the  best  acquisition  strategy  for  future  BMDS  targets.  It  is 
expected  to  be  completed  in  mid-2009.  Whether  or  not  MDA  decides  to 
restart  the  acquisition  of  the  52-inch  targets,  or  other  FTF  variants,  and  the 
nature  of  those  targets  depends  on  the  results  of  this  review. 

Currently,  MDA  has  narrowed  its  FTF  development  efforts,  focusing  on  a 
single  vehicle,  the  72-inch  LV-2  ground-launched  target.  The  first  launch 
was  supposed  to  determine  the  viability  of  the  FTF  concept  and  the 


2b  A  sound  business  case  demonstrates  that  (1)  the  identified  needs  are  real  and  necessary 
and  are  best  met  with  the  chosen  concept  and  (2)  the  chosen  concept  can  be  developed 
and  produced  with  existing  resources — such  as  technical,  knowledge,  funding,  time  and 
management  capacity. 

21  GAO,  Defense  Acquisitions:  Sound  Business  Case  Needed  to  Implement  Missile  Defense 
Agency’s  Targets  Program,  GAO-08-1113  (Washington,  D.C.:  Sept.  26,  2008). 
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feasibility  of  discontinuing  the  use  of  existing  targets.  However,  rather 
than  first  conducting  a  separate  developmental  test  to  confirm  the  target’s 
capability,  MDA  has  chosen  a  much  riskier  approach.  The  first  launch  of 
the  new  LV-2  target  will  be  in  an  Aegis  BMD  intercept  test.  Aegis  BMD 
originally  planned  to  use  this  new  target  in  a  fiscal  year  2008  flight  test; 
however,  because  the  target  was  not  ready,  the  test  is  delayed  until  at  least 
the  third  quarter  of  fiscal  year  2009. 

Repeated  target  problems  and  test  cancellations  have  also  affected  the 
development  of  capabilities  needed  to  discriminate  the  real  target  from 
countermeasures.  Without  opportunities  to  test  the  functionality  of  the 
software,  there  now  is  a  system-level  shortfall  in  BMDS  progress  toward 
developing  a  target  discrimination  capability  against  more  sophisticated 
countermeasures  in  the  midcourse  phase  of  flight.  In  order  to  improve  the 
effectiveness  of  the  BMDS  against  evolving  threats,  MDA  elements  are 
developing  advanced  discrimination  software  in  their  component’s 
sensors.  The  advanced  discrimination  software  is  critical  to  distinguish  the 
threat  re-entry  vehicle  from  associated  countermeasures  and  debris. 

Target  failures  during  tests  prevented  opportunities  to  gather  data  to 
assess  how  well  discrimination  software  performs  in  an  operational 
environment. 


Overall  Performance  of 
BMDS  Can  Not  Yet  Be 
Assessed 


MDA’s  modeling  and  simulation  program  enables  MDA  to  assess  the 
capabilities  and  limitations  of  how  BMDS  performs  under  a  wider  variety 
of  conditions  than  can  be  accomplished  through  the  limited  number  of 
flight  tests  conducted.  Flight  tests  alone  are  insufficient  because  they  only 
demonstrate  a  single  collection  data  point  of  element  and  system 
performance.  Flight  tests  are,  however,  an  essential  tool  used  to  both 
validate  performance  of  the  BMDS  and  to  anchor  the  models  and 
simulations  to  ensure  that  they  accurately  reflect  real  performance. 
Computer  models  of  individual  elements  replicate  how  those  elements 
function.  These  models  are  then  combined  into  various  configurations  that 
simulate  the  BMDS  engagement  of  enemy  ballistic  missiles. 
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To  ensure  confidence  in  the  accuracy  of  modeling  and  simulation  in 
representing  BMDS  capability,  the  program  goes  through  a  process  called 
accreditation.28  Element  models  are  validated  individually  using  flight  and 
other  test  data  and  accredited  for  their  intended  use.  MDA  intends  to 
group  these  models  into  system-level  representations  according  to  user 
needs.  One  such  grouping  is  the  annual  performance  assessment,  a 
system-level  end-to-end  simulation  that  assesses  the  performance  of  the 
current  BMDS  configuration.29  The  performance  assessment  integrates 
element-specific  models  into  a  coherent  representation  of  the  BMDS. 
Performance  assessments  are  used  to: 

•  assess  objectives  from  MDA’s  Deputy  of  Engineering  and  the  BMDS 
Operational  Test  Agency,30 

•  support  MDA  decisions  about  engagement  sequence  group  capability 
deliveries,  and 

•  support  MDA  decisions  about  BMDS  fielding  and  declaring 
capabilities. 

Fundamentally,  performance  assessments  anchored  by  flight  and  ground 
tests  are  a  comprehensive  means  to  fully  understand  the  performance 
capabilities  and  limitations  of  the  BMDS. 

Developing  an  end-to-end  system-level  model  and  simulation  has  been 
difficult.  BMDS  Operational  Test  Agency  officials  told  us  that  they  do  not 
anticipate  a  fully  accredited,  system-level  model  and  simulation  capability 
to  be  available  until  2011.  MDA’s  first  effort  to  bring  together  different 
element  models  and  simulations  to  produce  a  fully  accredited,  end-to-end 
model  and  simulation  for  Performance  Assessment  2007  was  unsuccessful 
primarily  because  of  inadequate  data  for  verification  and  validation  to 
support  accreditation  and  a  lack  of  common  threat  and  environment  input 
data  among  element  models.  Though  Performance  Assessment  2007  was  a 
success  in  establishing  a  capability  for  integrated  modeling  and  simulation 


"8  The  accreditation  of  models  and  simulations  is  an  official  certification  that  a  model  or 
simulation  is  acceptable  for  use  as  their  developers  intended.  Before  a  decision  to  accredit 
a  model,  MDA  must  first  verify  that  the  models  and  simulations  operate  as  the  designers 
conceptualized,  and  then  validate  that  the  models  are  sufficiently  accurate  representations 
of  real-world  conditions  for  their  intended  purposes. 

29  An  end-to-end  simulation  represents  a  complete  BMDS  engagement — from  enemy  missile 
launch  to  attempted  intercept  by  BMDS  kill  vehicle. 

30  The  BMDS  Operational  Test  Agency  provides  an  independent  accreditation  of  MDA 
models  and  simulations. 


Page  37 


GAO-09-338  Defense  Acquisitions 


in  a  short  time  frame,  it  was  unsuitable  to  assess  system-level  performance 
due  to  low  confidence  from  a  lack  of  accreditation.  Consequently,  acting 
on  a  joint  recommendation  between  MDA  and  the  Operational  Test 
Agency,  MDA  officials  canceled  their  2008  performance  assessment  efforts 
in  April  2008  because  of  developmental  risks  associated  with  modeling 
and  simulations,  focusing  instead  on  testing  and  models  for  Performance 
Assessment  2009.  MDA  officials  believe  that  the  refocused  efforts  will 
increase  the  chances  for  success  during  Performance  Assessment  2009. 

According  to  the  BMDS  Operational  Test  Agency’s  January  2009  Modeling 
and  Simulation  Accreditation  Report,  confidence  in  MDA’s  modeling  and 
simulation  efforts  remains  low  although  progress  was  made  during  the 
year.  MDA  is  now  exercising  stronger  central  leadership  to  provide 
guidance  and  resources  as  it  coordinates  the  development  of  verified  and 
validated  models  and  simulations,  as  recommended  by  a  2004  Defense 
Science  Board  study.  MDA  and  element  officials  are  now  working  more 
closely  with  the  BMDS  Operational  Test  Agency.  For  example,  MDA  and 
the  BMDS  Operational  Test  Agency  have  agreed  on  performance 
parameters  and  criteria  used  to  validate  element  models  and  simulations. 
Nonetheless,  BMDS  Operational  Test  Agency  officials  stated  that  there  are 
several  weaknesses  in  the  BMDS  testing  program  such  as: 

•  Insufficient  consideration  of  modeling  and  simulation  requirements  in 
MDA  flight  test  plans,  though  they  emphasized  that  MDA  is  finalizing  a 
list  of  such  parameters  for  future  flight  test  plans, 

•  Use  of  artificialities  in  flight  tests  which  limit  the  realism  of  scenarios 
for  anchoring  models  and  simulations,31  and 

•  Inadequate  test  planning  for  comprehensive  modeling  of  weather 
conditions.32 

MDA  intends  to  verify  and  validate  models  and  simulations  by  December 
2009  for  Performance  Assessment  2009.  However,  BMDS  Operational  Test 
Agency  officials  stated  that  there  is  a  high  risk  that  the  Performance 
Assessment  2009  analysis  will  be  delayed  because  of  remaining  challenges 
and  MDA’s  slow  progress  in  accreditation,  as  follows: 


31  The  BMDS  Operational  Test  Agency  defines  artificialities  as  BMDS  architecture,  targets, 
procedures,  and  conditions  that  exist  in  flight  tests  but  would  not  exist  in  the  real  world. 
Flight  test  artificialities  are  introduced  for  a  number  of  reasons,  such  as  increased  chances 
of  success,  range  safety,  data  collection,  and  asset  availability. 

32  Weather  conditions  include  rain,  clouds,  and  snow.  Severe  sea  states,  ice  loads,  or  winds 
could  render  tests  unsafe  to  execute. 
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•  The  compressed  schedule  of  ground  and  flight  tests  leaves  little  time 
for  data  analysis  that  is  essential  to  anchor  models  to  those  tests, 
particularly  for  a  complete  analysis  supporting  MDA’s  Performance 
Assessment  2009. 

•  Out  of  40  models,  the  BMDS  Operational  Test  Agency  recommended  in 
January  2009  full  accreditation  for  only  6  models,  partial  accreditation 
for  9  models,  and  no  accreditation  for  25  models. 

•  Because  MDA  canceled  the  follow-on  Performance  Assessment  2008, 
the  BMDS  Operational  Test  Agency  did  not  receive  verification  and 
validation  data  that  would  have  been  included  in  the  modeling  and 
simulation  portion  of  its  2008  operational  assessment. 

BMDS  Operational  Test  Agency  officials  told  us  that  MDA  also  does  not 
adequately  plan  for  the  collection  of  flight  test  data  and  post-flight 
reconstruction  to  support  anchoring  MDA  models  and  simulations,  even 
though  post-flight  reconstruction  is  needed  to  validate  that  models  and 
simulations  are  adequate  representations  of  the  real  world  for  their 
intended  purpose.33  MDA  guidance  emphasizes  that  one  of  the  primary 
objectives  of  the  MDA  ground  and  flight  test  program  is  to  anchor  BMDS 
models  and  simulations.  Additionally,  this  guidance  requires  MDA’s  testing 
program  to  work  with  the  MDA  engineers  to  define  a  test  program  that 
anchors  these  models  and  simulations  across  the  operating  spectrum. 
According  to  BMDS  Operational  Test  Agency  officials,  the  first  full  post¬ 
flight  reconstruction  was  conducted  in  December  2008. 

Despite  the  guidance  delineating  responsibilities  for  test  data,  MDA  test 
plans  currently  do  not  include  enough  detail  to  allocate  and  synchronize 
resources  in  order  to  anchor  models  and  simulations.  MDA  recently 
initiated  a  three-phase  review  of  the  entire  BMDS  test  program.  According 
to  MDA,  this  three-phase  review  will  emphasize  the  need  for  basing  BMDS 
test  planning  and  test  design  on  critical  factors  that  have  not  been  proven 
to  date  and  will  drive  target  selection  requirements.  One  outcome  of  the 
review  will  be  to  create  integrated  campaigns  of  ground  and  flight  tests  to 
efficiently  collect  data  needed  to  validate  the  models  and  simulations. 

MDA  intends  to  complete  all  three  phases  of  the  review  by  May  2009,  after 
which  MDA  intends  to  have  a  date  when  all  MDA  models  and  simulations 
will  be  verified  and  validated.  However,  the  current  lack  of  flight  test  data 
for  MDA’s  and  BMDS  Operational  Test  Agency  analysis  prevents  the  timely 


33  Post-flight  reconstruction  is  the  process  of  manually  recreating  and  running  a  past  flight 
test  scenario  in  a  simulated  environment. 
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validation  of  models  and  simulations  that  are  used  to  build  the  2009  end- 
to-end  performance  assessment. 


Production,  Fielding, 
and  Declaration  of 
Capabilities  Proceed 
despite  Delays  in 
Testing  and 
Assessments 


In  fiscal  year  2008,  MDA  met  most  of  its  delivery  goals.  However,  it 
continued  to  pursue  a  concurrent  development,  manufacturing  and 
fielding  strategy  in  which  assets  are  produced  and  fielded  before  they  are 
fully  demonstrated  through  testing  and  modeling.  Although  flight  tests  and 
modeling  and  simulation  produced  less  validation  of  performance  than 
planned,  MDA  continued  manufacturing  untested  components  and 
declaring  capabilities  ready  for  fielding.  For  example,  10  of  the  new 
configuration  kill  vehicles  for  the  GBI  will  have  been  manufactured  and 
delivered  before  being  flight-tested.  MDA  also  declared  that  it  had  fielded 
9  of  22  BMDS  capabilities  planned  for  2008  (postponing  13),  but  due  to  test 
cancellations  and  performance  assessment  delays,  it  had  to  change  the 
basis  of  these  declarations,  often  relying  on  previous,  less  realistic  testing. 


MDA  Met  Most  2008  Asset  MDA  achieved  four  of  the  five  delivery  goals  it  set  for  fiscal  year  2008  as 
Delivery  Goals  shown  in  the  table  8. 
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Table  8:  BMDS  Deliveries  and  Total  Fielded  Assets  as  of  September  30,  2008 


BMDS  element 

Fiscal  year  2008 
delivery  goals 

Assets  delivered 
in  fiscal  year  2008 

Total  assets  available 
(cumulative  total 
of  assets  since  2005) 

GMD 

3  interceptors 

0  interceptors 

24  interceptors5 

Sensors 

1  AN/TPY-2  radar 

Sea-based  X-band  radar 

1  AN/TPY-2  radar 

Sea-based  X-band  radar3 

4  AN/TPY-2  radars3 

Sea-based  X-band  radar 

Aegis  BMD 

20  SM-3  missiles 

20  SM-3  missiles 

34  SM-3  missiles 

15  destroyers 

3  cruisers 

C2BMC 

1  fielding  and  activation  site 

1  fielding  and  activation  site 

6  suites 

31  Web  browsers 

1  fielding  and  activation  site 

46  enterprise  workstations 

Source:  MDA  (data); 

GAO  (presentation). 

The  GMD  program  did  not  deliver  any  interceptors  as  planned  in  fiscal  year2008,  butwas  able  to 
deliver  two  interceptors— one  in  October  2008  and  one  in  November  2008.  Therefore,  the  cumulative 
total  for  G Bis  as  of  December  2008  is  26. 

“Partial  capability  for  the  Sea-based  X-band  radarwill  be  based  on  satisfying  planned  objectives  for 
two  tests  scheduled  for  fiscal  year  2009. 

AN/TPY-2  radars  were  formerly  known  as  Forward-Based  X-Band-  Transportable  radars. 
According  to  MDA,  an  additional  AN/TPY-2  radar  has  been  provided  and  is  undergoing 
Government  ground  testing. 

The  agency  planned  to  deliver  the  Sea-based  X-band  radar  and  three 
additional  GBIs  for  Block  1.0,  20  additional  SM-3  missiles  for  its  Block  2.0 
capability,  a  C2BMC  site  for  fielding  and  activation  for  its  Blocks  3. 1/3.2 
and  5.0  capabilities,  and  an  additional  AN/TPY-  2  radar.34  Although  partial 
capability  for  the  Sea-based  X-band  radar  will  not  be  declared  until  at  least 
fiscal  year  2009,  it  was  approved  for  Early  Capability  Delivery  in  fiscal  year 
2008.  The  agency  delivered  the  Aegis  BMD  SM-3  missiles,  the  AN/TPY-2 
radar,  and  the  C2BMC  site  in  fiscal  year  2008  as  planned,  but  was  unable 
to  deliver  the  GBIs  because  the  GMD  element  encountered  development 
challenges  with  components  for  the  CE-II  EKV.  In  addition,  the  Navy 
Commander,  Operational  Test  and  Evaluation  Force  declared  the  Aegis 
BMD  3.6  system  as  operationally  suitable  and  effective  in  October  2008. 
This  decision  signifies  that  18  Aegis  BMD-equipped  ships  and  90  SM-3 
missiles  are  ready  for  transition  to  the  Navy. 


34  AN/TPY-2  radars  were  formerly  known  as  Forward-Based  X-Band-Transportable  radars. 
According  to  MDA,  an  additional  AN/TPY-2  radar  has  been  provided  and  is  undergoing 
Government  ground  testing. 
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Production  and 
Fielding  of  BMDS 
Systems  Getting 
Ahead  of  Testing 


Despite  developmental  problems,  test  delays  and  MDA’s  inability  to 
complete  all  fiscal  year  2008  Director’s  test  knowledge  points, 
manufacturing,  production,  and  fielding  have  proceeded  close  to  schedule. 
In  some  cases  fielding  has  gotten  ahead  of  testing.  For  example,  Aegis 
BMD  expected  to  assess  the  ability  of  the  SM-3  Block  1A  missile  to  engage 
and  intercept  a  long-range  ballistic  target  to  satisfy  a  Director’s  test 
knowledge  point.  Even  though  that  test  has  been  delayed  until  the  third 
quarter  of  fiscal  year  2009,  MDA  purchased  20  SM-3  Block  lAs  in  fiscal 
year  2008. 

Furthermore,  MDA  intended  to  assess,  through  flight  tests,  the  CE-I  EKV’s 
capability  against  scenarios  that  included  complex  target  scenes  with 
countermeasures.  However,  due  to  the  frequent  restructuring  of  its  test 
plan  and  a  target  failure  during  its  most  recent  flight  test,  the  fielded 
configuration  for  GMD  has  not  completed  a  test  against  countermeasures. 
According  to  MDA,  no  more  CE-I  flight  tests  have  been  approved,  although 
the  agency  is  considering  additional  flight  testing  of  the  CE-I  EKV  in  the 
future.  Moreover,  earlier  ground  and  flight  testing,  along  with 
manufacturing  discoveries  prompted  the  GMD  program  to  initiate  a 
refurbishment  program  for  the  kill  vehicles  and  the  boosters. 
Refurbishment  consists  of:  (1)  reliability  improvements  to  address  high 
priority  risks  and  to  support  the  development  and  understanding  of  GBI 
reliability  and  (2)  surveillance  of  aging  through  the  examination  of 
removed  components.  Consequently,  the  capability  of  the  CE-I,  including 
improvements  designed  to  mitigate  risk,  as  well  as  understand  its 
capabilities  and  limitations  against  targets  employing  countermeasures 
may  not  be  flight-tested,  yet  all  24  interceptors  with  this  configuration  are 
already  emplaced  and  declared  operational. 

More  importantly,  the  GMD  program  continues  to  experience  test  delays, 
causing  fielding  to  outpace  flight  tests  as  shown  in  figure  4. 
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Figure  4:  GMD  Flight  Test  and  Fielding  Plan  for  Interceptors  Comparison — September  2006  versus  January  2009 


Source:  GAO  analysis  of  MDA  data. 


For  example,  the  program  has  only  been  able  to  conduct  two  intercepts 
since  2006  for  verifying  the  fielded  configuration  yet  the  production  of 
interceptors  continues.  According  to  GMD’s  September  2006  flight  test 
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plan,  for  fiscal  years  2007  and  2008,  and  the  first  quarter  of  fiscal  year  2009 
it  was  going  to  conduct  seven  flight  tests,  including  a  test  that  would 
utilize  2  GBIs  against  a  single  target — known  as  a  salvo  test35  — and  field  16 
new  GBIs.  By  January  2009  GMD  had  changed  its  plan,  removing  the  salvo 
test  and  conducting  two  flight  tests,  yet  it  fielded  13  GBIs. 

Similarly,  GMD  had  planned  to  conduct  an  intercept  test  to  assess  the 
enhanced  version  of  the  EKV  called  the  Capability  Enhancement  II  in  the 
first  quarter  of  fiscal  year  2008,  months  before  emplacing  any  interceptors 
with  this  configuration.  However,  developmental  problems  with  the  new 
configuration’s  inertial  measurement  unit  and  the  target  delayed  the  first 
flight  test  with  the  CE-II  configuration — FTG-06 — until  at  least  the  fourth 
quarter  of  fiscal  year  2009.  Despite  these  delays,  MDA  expects  to  have 
emplaced  five  CE-II  interceptors  before  this  flight  test.  MDA  indicated  that 
these  five  interceptors  will  not  be  declared  operational  until  the 
satisfactory  completion  of  the  test  and  the  Program  Change  Board 
declares  their  status.  However,  MDA  projects  that  10  CE-II  EKVs  will  have 
been  manufactured  and  delivered  before  that  first  flight  test  demonstrates 
the  CE-II  capability.  This  amounts  to  over  half  of  the  CE-II  EKV  deliveries 
that  are  currently  on  contract. 

MDA  did  not  emplace  the  three  GBIs  it  needed  to  meet  its  fiscal  year  2008 
fielding  goals.  MDA  will  have  to  emplace  twice  as  many  GBIs  than  planned 
in  fiscal  year  2009  before  Block  1.0  can  be  declared  complete.  As  of 
January  2009,  the  agency  had  emplaced  two  and  must  emplace  four  more 
in  order  to  complete  Block  1.0  as  planned. 

Major  defense  and  acquisition  programs  must  complete  operational  test 
and  evaluation  before  entering  full-rate  production.36  Because  MDA 
considers  the  assets  it  has  fielded  to  be  developmental,  it  has  not 
advanced  BMDS  elements  to  DOD’s  acquisition  cycle  or  begun  full-rate 
production.  Therefore,  MDA  has  not  yet  triggered  the  requirement  for  an 
operational  test  and  evaluation  prior  to  fielding.  However,  MDA’s 
concurrent  approach  to  developing  and  fielding  assets  has  led  to  testing 


35  The  National  Defense  Authorization  Act  for  Fiscal  Year  2002,  Pub.  L.  No.  107-107,  § 
234(c),  states  that  “for  ground-based  midcourse  interceptor  systems,  the  Secretary  of 
Defense  shall  initiate  steps  during  fiscal  year  2002  to  establish  a  flight  test  capability  of 
launching  not  less  than  three  missile  defense  interceptors  and  not  less  than  two  ballistic 
missile  targets  to  provide  a  realistic  test  infrastructure.”  Currently,  GMD  has  not  conducted 
this  test. 

36 10  U.S.C.  §  2399. 
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problems  and  concerns  about  the  performance  of  some  fielded  assets. 
After  two  flight  test  failures  in  2005,  MDA  undertook  a  Mission  Readiness 
Task  Force  to  establish  confidence  in  GMD’s  ability  to  reliably  hit  its 
target,  establish  credibility  in  setting  and  meeting  test  event  dates,  build 
increasing  levels  of  operationally  realistic  test  procedures  and  scenarios, 
raise  confidence  in  successful  outcomes  of  flight  missions,  and  conduct 
the  next  flight  test  as  soon  as  practical  within  acceptable  risk  bounds. 
However,  GMD  accelerated  the  objectives  for  its  test  program  after  the 
first  Mission  Readiness  Task  Force  flight  test  and  the  program  continues 
to  experience  developmental  problems  with  key  interceptor  components. 
MDA  also  separately  established  a  refurbishment  program  designed  to 
replace  questionable  interceptor  parts  and  increase  reliability  of  GBIs. 
Since  2006,  we  have  reported  that  the  performance  of  some  fielded  GBIs 
was  uncertain. 

Despite  MDA’s  previous  efforts  to  build  confidence  in  its  test  program, 
MDA  continues  to  pursue  a  risky  approach  for  fielding  BMDS  assets  under 
its  new  block  structure.  In  March  2008,  we  reported  that  MDA’s  new  block 
structure  did  not  address  whether  it  would  continue  its  practice  of 
concurrently  developing  and  fielding  BMDS  elements  and  components. 
However,  in  2008  the  agency  continued  to  field  assets  without  adequate 
knowledge.  MDA  emplaced  GBIs  during  the  year  although  its 
refurbishment  program  was  barely  underway,  meaning  that  the  risks  of 
rework  continue.  To  date,  26  GBIs  have  been  emplaced — many  of  which 
may  contain  unreliable  parts — and  only  a  few  have  been  refurbished  since 
the  initiation  of  the  refurbishment  program  in  2007.  According  to  program 
officials,  some  improvements  have  already  been  introduced  into  the 
manufacturing  flow  and  demonstrated  during  flight  testing. 

While  it  is  always  a  concern  when  tests  are  eliminated  or  the  complexity  of 
a  planned  test  is  reduced,  the  concern  is  heightened  for  a  system  of 
systems  such  as  the  BMDS  because  of  the  complex  interaction  of 
components  within  an  element,  and  between  that  element  and  the  other 
elements  within  the  BMDS.  Consequently,  the  need  to  synchronize  the 
development  and  testing  of  different  capabilities  is  crucial  before  fielding 
begins.  For  example,  for  certain  engagement  scenarios,  the  ground-based 
interceptor  will  launch  based  on  information  provided  by  an  entirely 
separate  element  such  as  an  Aegis  cruiser  or  destroyer.  If  a  problem  is 
discovered  during  these  flight  tests,  post-flight  reconstruction  using 
models  needs  to  be  conducted,  the  root-cause  must  be  determined,  a 
solution  or  mitigation  must  be  developed  and  implemented,  and  a  new  test 
to  confirm  the  effectiveness  of  the  solution  or  mitigation  must  be 
performed. 
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Reduced  Testing  Has 
Delayed  Some  Capability 
Declarations  and 
Weakened  the  Basis  for 
Others 


When  MDA  determines  that  a  capability  can  be  considered  for  operational 
use,  it  does  so  through  a  formal  declaration.  MDA  uses  an  incremental 
declaration  process  to  designate  BMDS  capability  for  its  blocks  in  three 
levels — early,  partial  and  full.  The  first  two  levels  allow  these  BMDS 
features  to  play  a  limited  role  in  system  operations  before  they  have 
attained  their  full  level  of  capability.  Each  capability  designation  in  the 
delivery  schedule  represents  upgraded  capacity  to  support  the  overall 
function  of  BMDS  in  its  mission  as  well  as  the  level  of  MDA  confidence  in 
the  system’s  performance.  Capability  declarations  are  important  because 
MDA  uses  them  to  assess  progress  toward  block  completion.  MDA 
guidance  calls  for  an  orderly  sequence  of  events  that  lead  to  declaring  that 
a  fielded  capability  has  been  achieved  and  is  ready  for  consideration  for 
operational  use. 

MDA  bases  its  declarations  on,  among  other  things,  a  combination  of 
models  and  simulations — such  as  end-to-end  performance  assessments — 
and  ground  tests  all  anchored  to  flight  test  data.  Because  performance 
assessments  analyze  the  BMDS  as  an  entire  system  in  a  variety  of  ways, 
they  provide  more  comprehensive  information  than  flight  tests  alone. 
These  events  and  assessments  build  on  each  other  every  year  as  MDA  adds 
capabilities  by  improving  hardware  and  software.  MDA  decision  makers 
would  then  declare  the  achievement  of  capability  goals  for  engagement 
sequence  groups  based  on  performance  assessments.  While  in  some 
instances,  declarations  of  capability  have  been  deferred,  in  other  instances 
MDA  has  declared  capabilities  despite  shortfalls  in  testing,  modeling  and 
simulation,  and  performance  assessments.  The  agency  declared  the 
delivery  of  nine  capabilities  during  fiscal  year  2008  as  shown  in  figure  5 
below.  It  declared  three  early  capabilities  for  Block  1.0  engagement 
sequence  groups,  three  early  as  well  as  one  full  capability  for  Block  2.0 
engagement  sequence  groups,  and  one  early  capability  and  one  partial 
capability  for  Block  3. 1/3.2  engagement  sequence  groups. 
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Figure  5:  Timeline  Showing  Declaration  of  Capabilities  in  Fiscal  Year  2008 


FY08 

01  I  02  1  03  1  04 

Block  1.0 

GBI  Launch-on  COBRA 

DANE  radar  (Beale) 

GBI  Engage-on  sea-based 
X-band  radar 

GBI  Launch-on  sea-based 
X-band  radar 

A - A 

A - A 

A - A 

Block  2.0 

SM-3  Engage-on 
shipboard  Aegis  radar 

SM-3  Launch  on  remote 
shipboard  Aegis  radar 

SM-2  Engage-on 
shipboard  Aegis  radar 

THAAD  Engage-on 

AN/TPY-2  radar 
(terminal  mode) 

A - A 

A - A 

A - A 

A - A 

Block  3.1  /3.2 

GBI  Engage  on 

COBRA  DANE  radar 

Mod  1  (Fylingdales, 

UK;  forward-based 
(AN/TPY-2  radar)) 

GBI  Launch  on  shipboard 
Aegis  radar  Mod  1 
(Fylingdales,  UK;  sea-based 
X-band  radar) 

A 

A - A 

Capability 

Milestone 

Milestone  Delayed 

declaration 

scheduled 

achieved 

Early 

A 

A 

Partial 

A 

A 

Full 

A 

A 

Source:  GAO  analysis  ofMDA  data. 

Note:  Our  analysis  above  is  based  on  the  MDA  Master  Execution  Fielding  Schedule  dated  October 
2007  as  well  as  the  Master  Fielding  Plan  dated  February  2008.  Commensurate  with  its  new  block 
structure,  MDA  reported  a  subset  of  these  as  part  of  its  fiscal  year  2008  Statement  of  Goals  dated 
J  anuary  2008.  However,  MDA  continues  to  work  toward  declaring  all  of  the  October  2007 
engagementsequence  groups. 
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MDA  had  intended  to  use  the  results  of  a  flight  test  (FTG-04),  that  was 
later  canceled; 37  a  distributed  ground  test  (GTD-03),  that  was  delayed  into 
fiscal  year  2009; 38  and  the  results  of  Performance  Assessments  2007  and 
2008  to  determine  if  capabilities  were  ready  for  declaration  in  fiscal  year 
2008.  Consequently,  these  shortfalls  in  knowledge  led  MDA  to  reduce  the 
basis  for  declaring  capability  goals.  Performance  Assessment  2007 — 
identified  by  MDA  as  a  key  source  to  assess  capabilities  during  fiscal  year 
2008 — achieved  only  limited  accreditation.39  This  less-than-full 
accreditation  indicated  that  MDA  could  not  rely  on  the  assessment’s 
results  to  gauge  end-to-end  BMDS  performance.  Subsequently,  MDA 
officials  decided  to  cancel  Performance  Assessment  2008  because  they 
needed  time  to  address  problems  and  prepare  for  Performance 
Assessment  2009. 

While  MDA  officials  declared  these  capabilities  during  fiscal  year  2008, 
they  did  so  after  mostly  reducing  the  basis  for  the  declarations.  They 
reverted  back  in  several  cases  to  older  ground  and  flight  tests,  though 
MDA  in  a  few  cases  added  some  newer  flight  and  ground  tests  as  well.  For 
example,  MDA  declared  early  Block  1.0  capability  for  three  engagement 
sequence  groups  in  fiscal  year  2008  without  the  planned  results  from 
Performance  Assessment  2007.  In  all  cases,  though  MDA  had  intended  to 
use  the  final  results  from  comprehensive  performance  assessments,  after 
revising  the  basis  for  declaring  capability  goals  it  eliminated  them  entirely. 
Specifically,  MDA  dropped  some  sources  of  data  it  expected  to  use,  such 
as  the  canceled  Performance  Assessment  2008,  and  shifted  from  flight  and 
ground  tests  planned  to  occur  in  fiscal  year  2008  to  older  flight  and  ground 
tests.  For  example,  in  Block  2.0  MDA  declared  full  capability  during  fiscal 
year  2008  for  one  engagement  sequence  group,  Aegis  BMDS  engage  on  its 
shipboard  radar,  even  though  Performance  Assessment  2008  had  been 
canceled.  MDA  instead  based  its  decision  on  integrated  and  distributed 
ground  tests  (GTI-02  and  GTD-02)  conducted  in  calendar  year  2007  as  well 
as  prior  flight  tests  during  fiscal  years  2006  through  2008.  However,  the 


37  See  app  III  which  details  reasons  for  the  FTG-04  cancellation. 

38  GTD-03  was  delayed  to  support  warfighter  needs  and  resulted  in  delayed  capability 
assessments  and  capability  declaration  later  than  planned. 

39  The  accreditation  of  models  and  simulations  is  an  official  certification  that  a  model  or 
simulation  is  acceptable  for  use  as  their  developers  intended.  Before  a  decision  to  accredit 
a  model,  MDA  must  first  verify  that  the  models  and  simulations  operate  as  the  designers 
conceptualized,  and  then  validate  that  the  models  are  sufficiently  accurate  representations 
of  real-world  conditions  for  their  intended  purposes. 
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BMDS  Operational  Testing  Agency  raised  concerns  about  the 
comprehensiveness  of  the  GTI-02  scenarios,40  specifically,  the  incorrect 
configuration  of  U.S.  satellites  and  threat  data. 

MDA  also  deferred  13  capability  goals  scheduled  to  occur  in  fiscal  year 
2008  to  the  end  of  fiscal  year  2009,  as  shown  in  figure  6  below. 


40  GTI-02  included  models  and  simulations. 
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Figure  6:  Timeline  Showing  Deferred  Declaration  of  Capabilities  from  Fiscal  Year  2008  to  2009 
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Note:  Our  analysis  above  is  based  on  the  MDA  Master  Execution  Fielding  Schedule  dated  October 
2007  as  well  as  the  Master  Fielding  Plan  dated  February  2008.  Commensurate  with  its  new  block 
structure,  MDA  reported  a  subset  of  these  as  part  of  its  fiscal  year  2008  Statement  of  Goals  dated 
J  anuary  2008.  Flowever,  MDA  continues  to  work  toward  declaring  all  of  the  October  2007 
engagementsequence  groups.  Several  engagementsequence  groups  are  notshown  here  because 
they  are  Block  3.3,  which  MDA  has  not  yet  baselined. 

MDA  intended  to  declare  all  Block  1.0  engagement  sequence  groups  as 
fully  capable  by  the  middle  of  fiscal  year  2009.41  However,  as  MDA 
encountered  test  delays  and  technical  challenges,  it  had  to  defer  full 
capability  declaration  for  these  engagement  sequence  groups  until  the  end 
of  fiscal  year  2009.  For  Block  2.0,  MDA  also  deferred  declaring  full 
capability  for  one  of  the  two  planned  full  capability  declarations  for  fiscal 
year  2008.  This  declaration  is  contingent  upon  the  review  of  a  ground  test 
that  has  been  rescheduled  to  the  second  quarter  of  fiscal  year  2009  and  a 
flight  test  rescheduled  to  the  third  quarter  of  fiscal  year  2009.  MDA  also 
deferred  one  full  and  two  early  capability  declarations  for  Block  3. 1/3.2 
beyond  the  end  of  fiscal  year  2009. 

In  response  to  the  limitations  of  Performance  Assessment  2007,  the 
cancellation  of  Performance  Assessment  2008  and  FTG-04,  and  the 
delayed  GTD-03  and  FTG-05  flight  tests,  MDA  is  planning  to  rely  on  older 
ground  and  flight  tests;  a  sensor  flight  test,  FTX-03,  instead  of  intercept 
flight  tests;  and  the  initial  quick  look  review  of  Performance  Assessment 
2009  instead  of  the  previously  planned  full  analysis.  Appendix  IV  provides 
a  detailed  layout  for  the  reduced  basis  of  capability  declarations  for  fiscal 
years  2008  and  2009. 

Since  MDA  was  only  able  to  declare  a  few  of  the  capabilities  it  planned  for 
fiscal  year  2008,  the  schedule  for  fiscal  year  2009  and  subsequent  years 
will  be  compressed  if  the  agency  plans  to  maintain  the  schedule  it  has  set 
for  its  blocks.  For  example,  MDA  may  need  to  declare  three  times  as  many 
capabilities  than  originally  planned  for  fiscal  year  2009  in  order  to  meet 
the  2009  capability  declaration  schedule.  In  addition,  if  the  schedule 
cannot  be  maintained,  MDA  will  likely  have  to  make  further  adjustments 
to  mitigate  additional  delays  in  BMDS  capabilities. 

Increased  reliance  on  integrated  ground  testing  will  provide  less 
knowledge  than  a  complete  analysis  of  capabilities  from  a  performance 


41  Our  assessment  of  engagement  sequence  groups  utilizes  MDA’s  plans  as  of  October  1, 
2007.  Commensurate  with  the  new  block  structure,  MDA  devised  a  baseline  in  February 
2008  for  engagement  sequence  groups  for  Blocks  1.0,  2.0,  3.1,  and  3.2. 
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assessment.  Integrated  ground  testing  involves  less  robust  conditions  than 
distributed  ground  testing,  which  involves  operational  systems  in  the  field. 
The  MDA  master  fielding  plan  indicates  that  the  agency  originally  intended 
to  take  a  more  comprehensive  approach  upon  which  to  base  capability 
declarations.  Reliance  on  an  upcoming  Performance  Assessment  2009 
quick  look  for  Block  1.0  completion  is  a  particular  concern  because  the 
knowledge  it  provides  may  be  limited  with  respect  to  testing,  according  to 
BMDS  Operational  Test  Agency  officials.  For  example,  officials  told  us 
that  a  quick  look  may  indicate  anomalies  from  a  test  but  will  not  analyze 
their  causes.  In  contrast,  MDA  originally  planned  to  have  a  complete 
analysis  from  the  Performance  Assessment  2009  models,  simulations,  and 
tests. 


Limited  Progress 
Made  in  Improving 
Transparency  and 
Accountability 


In  March  2008,  we  reported  that  efforts  were  underway  to  improve  BMDS 
management,  transparency,  accountability,  and  oversight  including  a  new 
executive  board  outside  of  MDA  and  a  new  block  structure  along  with 
other  improvements  within  MDA.42  Since  that  time,  the  executive  board 
that  was  established  in  2007  has  acted  with  increased  oversight.  MDA’s 
efforts,  however,  have  not  made  the  expected  progress.  In  particular,  MDA 
has  decided  to  retain  the  option  of  deferring  work  from  one  block  to 
another;  cost  baselines  have  not  been  established;  test  baselines  remain 
relatively  unstable;  and  requesting  procurement  funds  for  some  assets,  as 
directed  by  Congress,  will  not  occur  until  fiscal  year  2010.43 

To  accomplish  its  mission,  in  2002  the  Secretary  of  Defense  gave  MDA 
requirements,  acquisition,  and  budget  flexibilities  and  relief  from  some 
oversight  mechanisms  and  reporting  responsibilities.  The  flexibility 
granted  to  MDA  has  allowed  concurrent  development,  testing, 
manufacturing,  and  fielding.  MDA  used  this  flexibility  to  quickly  develop 
and  field  the  first  increment  of  capability  in  2005.  In  August  2008,  in 
response  to  Congressional  direction  to  assess  the  current  and  future 
missions,  roles,  and  structure  of  MDA,  an  independent  study  group  agreed 
that  there  is  a  need  to  move  MDA  toward  more  normal  acquisition 
processes.  However,  the  group  noted  that  the  continuous  evolution  of  the 


42  GAO-08-448. 

43  Section  223(b)  of  the  National  Defense  Authorization  Act  of  2008,  Pub.  L.  No.  110-181 
(Jan.  28,  2008)  specified  a  revised  budget  structure  of  the  missile  defense  budget  to  be 
submitted  in  the  President’s  budget  no  later  than  the  first  Monday  in  February.  31  U.S.C. 
§1105. 
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BMDS  requires  that  the  approach  to  setting  requirements  for,  developing, 
and  fielding  increments  of  capability  should  remain  as  special  authorities 
with  oversight  of  the  Missile  Defense  Executive  Board  (MDEB).  Further, 
in  regards  to  budget  flexibilities,  the  independent  group  concluded  that 
while  these  flexibilities  may  have  been  deemed  necessary  at  the  time,  it 
should  not  have  been  expected  that  all  the  special  authorities  granted  to 
MDA  would  continue  or  would  have  a  need  to  continue  in  full  force 
beyond  achieving  the  President’s  goal  of  deploying  a  set  of  initial 
capabilities. 


Missile  Defense  Executive  During  2008,  the  MDEB  appeared  to  act  with  an  increased  level  of 
Board’s  Oversight  Role  authority  in  providing  oversight  of  MDA  and  the  BMDS.  For  example,  the 

More  Active  in  2008  board  took  on  a  major  role  in  making  key  decisions  regarding  the 

transition  of  elements  to  military  services.  We  previously  reported  that 
MDA  and  the  military  services  had  been  negotiating  the  transition  of 
responsibilities  for  the  sustainment  of  fielded  BMDS  elements,  but  this 
process  had  been  proven  to  be  arduous  and  time  consuming.  However,  in 
2008,  with  the  influence  of  the  MDEB,  a  lead  military  service  designation 
was  appointed  for  one  BMDS  asset — the  Sea-based  X-band  radar.44 

In  March  2008,  we  reported  that  the  MDEB  could  play  a  key  role  in  the 
Joint  Requirements  Oversight  Council’s  proposal  to  return  the  BMDS  to 
the  Joint  Capabilities  Integration  and  Development  System  requirements 
process — a  formal  DOD  procedure  followed  by  most  DOD  programs  that 
defines  acquisition  requirements  and  evaluation  criteria  for  future  defense 
programs.  In  responding  to  the  proposal,  the  Acting  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  recommended  that  the 
Deputy  Secretary  of  Defense  delay  his  approval  of  the  Joint  Staffs 
proposal  until  the  MDEB  could  review  the  proposal  and  provide  a 
recommendation.  According  to  Acquisition,  Technology  and  Logistics 
officials,  no  decision  has  been  made  regarding  returning  the  BMDS  to  the 
requirements  process.  However,  the  Deputy  Secretary  of  Defense,  in 
September  2008,  appeared  to  strengthen  the  oversight  role  of  the  MDEB, 
clarifying  the  roles  of  the  MDEB  as  well  as  MDA,  the  Office  of  the 
Secretary  of  Defense,  Combatant  Commands,  and  Military  Departments. 
With  respect  to  the  role  of  the  MDEB,  he  established  a  life  cycle 
management  process  for  the  BMDS  stating  that  the  MDEB  will  recommend 


44  Lead  services  have  already  been  designated  for  Aegis  BMD,  the  AN/TPY-2  radar,  THAAD, 
GMD,  ABL,  the  European  radar,  Cobra  Dane,  and  upgraded  early  warning  radars. 
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and  oversee  implementation  of  strategic  policies  and  plans,  program 
priorities,  and  investment  options  to  protect  our  Nation  and  our  allies 
from  missile  attack.  One  of  the  MDEB  functions  is  to  provide  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics — or 
Deputy  Secretary  of  Defense,  as  necessary — a  recommended  strategic 
program  plan  and  feasible  funding  strategy  for  approval.  The  Deputy 
Secretary  further  noted  that,  through  the  use  of  the  BMDS  Life  Cycle 
Management  Process  outlined  in  the  memo,  the  MDEB  will  oversee  the 
annual  preparation  of  the  BMDS  portfolio,  including  BMDS-required 
capabilities  and  a  program  plan  to  meet  the  requirements  with  Research, 
Development  Test  &  Evaluation,  procurement,  operations  and 
maintenance,  and  military  construction  resources  in  defense-wide 
accounts. 

To  further  increase  BMDS  transparency  and  oversight  of  the  BMDS,  the 
Under  Secretary  of  Defense  Acquisition,  Technology,  and  Logistics  plans 
to  hold  program  reviews  for  several  BMDS  elements  commensurate  with 
the  authority  granted  to  the  MDEB  by  the  Deputy  Secretary  of  Defense. 
According  to  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  officials,  the  MDEB  conducted  its  first  of  such  reviews  in 
November  2008  of  the  THAAD  program.  This  review  covered  production 
and  the  element’s  contract  schedule.  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  officials  told  us  that  these  reviews 
are  designed  to  provide  the  Deputy  Director  Acquisition,  Technology,  and 
Logistics  with  comprehensive  information  that  will  be  used  as  the  basis  for 
MDEB  recommendations  for  the  BMDS  business  case  and  baseline 
processes —  a  process  which,  according  to  these  officials,  is  similar  to  the 
traditional  Defense  Acquisition  Board  process  for  reviewing  other  major 
acquisition  programs.  However,  it  is  unclear  whether  the  information 
provided  to  the  MDEB  will  be  comparable  to  that  produced  for  other 
major  acquisition  program  reviews  as  most  of  the  information  appears  to 
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be  derived  or  presented  by  MDA  as  opposed  to  independent  sources  as 
required  for  traditional  major  defense  acquisition  programs.46 

Efforts  to  Improve 
Transparency  of  MDA’s 
Work  Have  Not  Progressed 
as  Planned 

Deferral  of  Work  In  2007,  MDA  redefined  its  block  structure  to  better  communicate  its  plans 

and  goals  to  Congress.  The  agency’s  new  structure  is  based  on  fielding 
capabilities  that  address  particular  threats  instead  of  the  biennial  time 
periods  previously  used  to  develop  and  field  the  BMDS.  Last  year,  we 
reported  that  MDA’s  new  block  plans  included  many  positive  changes.46 
However,  MDA,  with  its  submission  of  its  Fiscal  Year  2008  Statement  of 
Goals,  reversed  some  of  the  positive  aspects  of  the  new  block  structure. 
For  example,  we  previously  reported  that  the  new  block  structure  would 
improve  the  transparency  of  each  block’s  actual  cost  by  disallowing  the 
deferral  of  work  from  one  block  to  another.  Under  its  prior  block 
structure,  MDA  deferred  work  from  one  block  to  another;  but  it  did  not 
track  the  cost  of  the  deferred  work  so  that  it  could  be  attributed  to  the 
block  that  it  benefited.  Because  MDA  did  not  track  the  cost  of  the  deferred 
work,  the  agency  was  unable  to  adjust  the  cost  of  its  blocks  to  accurately 
capture  the  cost  of  each.  This  weakened  the  link  between  budget  funds 
and  the  work  performed.  Last  year,  MDA  officials  told  us  that  under  its 
new  block  approach,  MDA  would  no  longer  transfer  work  under  any 
circumstances  to  a  different  block.  However,  MDA  officials  recently  said 
that  they  are  retaining  the  option  to  move  work  from  one  block  to  another 
as  long  as  it  is  accompanied  by  a  rebaseline.  This  change  allows  the 


45  Before  a  program  can  enter  the  system  development  and  demonstration  phase  of  the 
acquisition  cycle,  statute  requires  that  certain  information  be  developed.  10  U.S.C.  §  2366b. 
In  2002,  the  Secretary  of  Defense  deferred  the  application  of  some  of  DOD’s  acquisition 
processes  to  BMDS.  Therefore,  MDA  has  not  yet  entered  System  Development  and 
Demonstration  which  would  trigger  the  statutes  requiring  the  development  of  information 
that  the  Defense  Acquisition  Board  uses  to  inform  its  decisions.  Most  major  defense 
acquisition  programs  are  also  required  by  statute  to  obtain  an  independent  verification  of 
program  cost  prior  to  beginning  system  development  and  demonstration,  and/or 
production  and  deployment.  10  U.S.C.  §  2434.  Statute  also  requires  an  independent 
verification  of  a  system’s  suitability  for  and  effectiveness  on  the  battlefield  before  a  major 
defense  acquisition  program  can  proceed  beyond  low-rate  initial  production.  10  U.S.C.  § 
2399. 

46  GAO-08-448. 
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Use  of  Procurement  Funds 


agency  to  continue  the  practice  of  moving  work  from  one  block  to 
another,  which  thereby  reduces  the  transparency  of  the  new  block 
structure  and  undermines  any  baselines  that  are  established. 

In  March  2007,  we  reported  that  the  majority  of  MDA’s  funding  comes 
from  the  Research,  Development,  Test,  and  Evaluation  appropriation 
account,  another  flexibility  provided  by  law.47  In  past  years,  Congress 
authorized  MDA  to  pay  for  assets  incrementally  using  research  and 
development  funds.  This  allowed  MDA  to  fund  the  purchase  of  assets  over 
multiple  years.  Congress  recently  restricted  MDA’s  authority  and  required 
MDA  to  purchase  certain  assets  with  procurement  funds  and  directed  that 
for  any  year  after  fiscal  year  2009,  MDA’s  budget  materials  must  delineate 
between  funds  needed  for  research,  development,  test,  and  evaluation, 
procurement,  operations  and  maintenance,  and  military  construction. 
Requiring  MDA  to  request  funding  in  these  appropriation  categories  will 
mean  that  it  will  be  required  to  follow  the  funding  policies  for  each 
category.  For  example,  using  procurement  funds  will  mean  that  MDA  will 
be  required  to  ensure  that  assets  are  fully  funded  in  the  year  of  their 
purchase,  rather  than  incrementally  funded  over  several  years. 

Congress  directed  in  the  2008  National  Defense  Authorization  Act,  for  any 
year  after  fiscal  year  2009,  that  MDA’s  budget  materials  delineate  between 
funds  needed  for  research,  development,  test,  and  evaluation, 
procurement,  operations  and  maintenance,  and  military  construction.  We 
have  previously  reported  that  using  procurement  funds  will  mean  that 
MDA  generally  will  be  required  to  adhere  to  congressional  policy  that 
assets  be  fully  funded  in  the  year  of  their  purchase,  rather  than 
incrementally  funded  over  several  years.  The  Congressional  Research 
Service  reported  in  2006  that  “incremental  funding  fell  out  of  favor 
because  opponents  believed  it  could  make  the  total  procurement  costs  of 
weapons  and  equipment  more  difficult  for  Congress  to  understand  and 
track,  create  a  potential  for  DOD  to  start  procurement  of  an  item  without 
necessarily  stating  its  total  cost  to  Congress,  permit  one  Congress  to  ‘tie 
the  hands’  of  future  Congresses,  and  increase  weapon  procurement  costs 
by  exposing  weapons  under  construction  to  uneconomic  start-up  and  stop 
costs.”48 


47  GAO-07-387. 

48  Congressional  Research  Service,  Defense  Procurement:  Full  Funding  Policy — 
Background,  Issues,  and  Options  for  Congress  (Oct.  20,  2006). 
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In  the  2008  National  Defense  Authorization  Act,  Congress  also  provided 
MDA  with  the  authority  to  use  procurement  funds  for  fiscal  years  2009  and 
2010  to  field  its  BMDS  capabilities  on  an  incremental  funding  basis, 
without  any  requirement  for  full  funding.  Congress  has  granted  similar 
authority  to  other  DOD  programs.  In  the  conference  report  accompanying 
the  Act,  Conferees  cautioned  DOD  that  additional  authority  will  be 
considered  on  a  limited  case-by-case  basis  and  noted  that  they  expect  that 
future  missile  defense  programs  will  be  funded  in  a  manner  more 
consistent  with  other  DOD  acquisition  programs. 

MDA  did  not  request  any  procurement  funds  for  fiscal  year  2009.  During 
our  audit,  the  agency  had  not  yet  released  the  2010  budget  request  to 
include  such  funding  categories.  However,  MDA  officials  told  us  that  the 
agency  plans  to  request  procurement  funds  for  some  of  its  assets  in  its 
fiscal  year  2010  request,  but  could  not  elaborate  on  its  plans  to  do  so. 

Given  that  data  was  unavailable,  it  is  unclear  for  which  assets 
procurement  funds  will  be  requested  or  the  extent  to  which  the  request 
will  meet  the  direction  given  by  Congress.  According  to  MDA  officials, 
information  regarding  its  plans  to  request  procurement  funding  will  not  be 
released  until  spring  2009. 

Baselines  Baselines  represent  starting  points  against  which  actual  progress  can  be 

measured.  They  are  thus  used  to  provide  indications  of  when  a  program  is 
diverting  from  a  plan.  Baselines  can  be  established  to  gauge  progress  in 
different  areas,  including  cost,  schedule  and  testing.  Overall,  the  BMDS 
does  not  have  baselines  that  are  useful  for  oversight.  With  regard  to  cost, 
we  have  already  discussed  the  lack  of  total  and  unit  cost  baselines  for 
missile  defense  as  well  as  the  frequency  of  changes  in  contract  baselines. 
MDA  made  some  progress  with  developing  a  schedule  baseline  for  its 
blocks  and  their  associated  capabilities.  The  agency’s  annual  Statement  of 
Goals  identifies  its  schedule  baseline  as  the  fiscal  year  dates  for  early, 
partial,  and  full  capability  deliveries  of  hardware  and  functionality  for  a 
block.  Thus,  while  MDA  has  changed  its  schedule  for  making  declarations, 
the  effect  of  the  change  can  be  determined  by  comparison  with  the 
original  schedule. 

MDA  does  not  have  test  baselines  for  its  blocks.  The  agency  does  however, 
have  baselines  for  its  test  program,  but  revises  them  frequently.  They  are 
therefore  not  effective  for  oversight.  The  agency  identified  its  Integrated 
Master  Test  Plan  as  the  test  baseline  for  the  BMDS.  However,  as  depicted 
in  table  9,  the  agency  has  made  a  number  of  changes  to  the  content  of  the 
baseline. 
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Table  9:  MDA  BMDS  Test  Baseline  Revisions 

Version 

Revisions/ 
change  date 

Rationale  for  change 

Version 

approved 

5.6.2 

February  20 

Interim  Update:  Changed  for  signature 

X 

8.01 

August  15 

Incorporated  MDA  new  block  construct; 
migrated  from  calendar  year  format  to 
fiscal  year  format. 

8.04 

October  12 

Updated  funding  status.  Incorporated 
Operational  Test  Agency  input. 

8.06 

November  6 

Preparation  for  internal  MDA 
coordination. 

8.07 

December  11 

Program  Change  Board  changes 
incorporated.  Includes  some  fiscal  year 
2008  and  2009  budget  decisions. 

8.1 

February  5 

Updated  with  Program  Change  Board 
changes 

X 

8.4 

J  uly  19 

Quarterly  update  based  on  Program 
Change  Board  and  working  group 
decisions 

9.1 

September  26 

Quarterly  update  limited  to  schedules. 

Source:  GAO  analysis  of  MDA  documents. 


The  official  approved  test  baseline  changes  every  year  and  there  are 
numerous  more  informal  changes  that  happen  more  frequently.  Most  of 
the  annual  revisions  to  the  test  baseline  occur  either  because  MDA  has 
changed  the  substance  of  test,  changed  the  timing  of  tests,  or  added  tests 
to  the  baseline. 

The  Integrated  Master  Test  Plan  establishes  the  executable  test  program 
for  the  current  fiscal  year  and  extends  through  the  following  fiscal  year. 
According  to  MDA,  the  test  plan  is  updated  quarterly  based  upon  decisions 
from  MDA  leadership  or  formal  decision-making  forums  such  as  the 
Program  Control  Board,  and  is  also  coordinated  annually  with  external 
agencies  such  as  the  Office  of  the  Director  of  Operational  Test  and 
Evaluation.  However,  as  shown  in  table  9,  there  are  several  versions  for  a 
given  quarter  and  as  many  as  seven  versions  have  been  developed  since 
the  fiscal  year  2008  baseline  was  established.  It  is  unclear  which  Integrated 
Master  Test  Plan  version  MDA  manages  to  at  any  given  time.  For  example, 
in  November  2008,  we  requested  the  latest  version  of  the  Integrated  Master 
Test  Plan  and  were  told  that  the  latest  approved  version  was  8.4,  which 
was  revised  in  July  2008.  However,  the  signature  page  for  that  version  is 
from  a  prior  version — version  8.1.  Since  there  is  no  signature  page 
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referring  to  version  8.4,  it  appears  that  this  version  is  unapproved  though 
MDA  officials  told  us  that  it  was  being  used  to  manage  the  BMDS. 

Agency  officials  maintain  that  the  document  is  used  to  manage  tests  and 
associated  requirements  for  key  test  events.  However,  it  is  unclear  how 
well  the  baseline  is  integrated  with  other  key  aspects  of  testing  such  as  the 
acquisition  of  targets  needed  to  execute  tests.  For  example,  in  some 
instances,  targets  are  placed  on  contract  for  two  or  more  years  in  advance 
of  the  planned  tests.  Yet,  the  test  baseline — the  Integrated  Master  Test 

Plan — does  not  appear  to  include  events  beyond  the  following  fiscal  year 
that  are  key  to  the  BMDS  test  program.  As  we  reported  in  September  2008, 
MDA  officials  acknowledged  that  its  target  contracts  did  not  capture  all 
element  testing  requirements  and  target  baselines  were  not  always 
established  before  targets  contracts  were  signed.49 

Conclusions 

In  2002,  MDA  was  given  unusual  authorities  to  expedite  the  fielding  of  an 
initial  BMDS  capability.  As  this  initial  capability  was  fielded  in  2005,  it 
showed  the  benefits  of  these  flexibilities.  MDA  has  improved  on  this 
capability  in  the  ensuing  years,  including  2008,  the  focus  of  this  report. 
Today,  the  program  is  still  operating  at  a  fast  pace,  as  production  and 
fielding  of  assets  outstrips  the  ability  to  test  and  validate  them.  A  collateral 
effect  of  these  flexibilities  has  been  reduced  visibility  into  actual  versus 
planned  progress.  Some  fundamental  questions  of  an  oversight  nature  are 
not  yet  answerable.  For  example,  a  comparison  of  actual  versus  planned 
costs  at  the  system  or  asset  level  is  not  yet  possible,  nor  is  an  assessment 
of  the  performance  of  the  fielded  system  as  a  whole.  Beginning  in  2007, 
MDA  began  efforts  to  improve  visibility  into  its  actual  performance, 
beginning  with  the  new  way  of  defining  blocks,  coupled  with  DOD’s 
creation  of  the  MDEB.  However,  progress  has  been  slow  in  some  areas 
and  value  for  money  cannot  be  satisfactorily  assessed.  Delays  are 
especially  important  in  a  program  of  this  size,  as  a  year  delay  in 
establishing  cost  baselines  means  another  $8  billion  to  $10  billion  may  be 
spent  in  the  meantime. 

With  the  transition  to  a  new  administration,  the  deployment  and 
subsequent  improvement  of  an  initial  capability,  a  new  agency  Director, 
and  a  new  block  structure  for  managing  the  BMDS,  an  opportunity  exists 
to  revisit  and  strengthen  the  processes  by  which  MDA  operates.  Looking 

49  GAO-08-1113. 
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to  the  future,  decision  makers  in  Congress  and  DOD  face  multi-billion 
dollar  investment  decisions  in  allocating  funds  both  within  MDA  and 
between  MDA  and  other  DOD  programs.  At  this  point,  a  better  balance 
must  still  be  struck  between  the  information  Congress  and  DOD  need  to 
conduct  oversight  of  the  BMDS  and  the  flexibility  MDA  needs  to  manage 
across  the  portfolio  of  elements  that  collectively  constitute  the  system’s 
capability. 

At  this  point,  the  balance  does  not  provide  sufficient  information  for 
effective  oversight.  In  particular: 

•  Total  cost  and  unit  cost  baselines  have  not  been  set  and  contract 
baselines  are  subject  to  frequent  changes.  Even  if  such  baselines  are 
set  as  planned,  they  will  only  capture  about  26  percent  of  MDA’s  work. 

•  Less  testing  is  conducted  than  planned,  thus  delaying  the  validation  of 
the  models  and  simulations  needed  to  assess  the  overall  performance 
of  the  BMDS.  Moreover,  test  plans  do  not  hold  and  are  revised  often,  in 
many  cases  due  to  the  poor  performance  of  target  missiles.  The 
current  test  plan  is  at  risk  given  its  ambitious  scope. 

•  Manufacturing,  production,  and  fielding  are  outpacing  testing, 
modeling,  and  validation.  Consequently,  fielding  decisions  and 
capability  declarations  are  being  made  with  limited  understanding  of 
system  effectiveness. 


We  recommend  that  the  Secretary  of  Defense  direct  the  MDEB  to  assess 
how  the  transparency  and  accountability  of  MDA’s  acquisitions  can  be 
strengthened  to  enhance  oversight,  such  as  by  adopting  relevant  aspects  of 
DOD’s  normal  requirements,  acquisition  and  budgeting  processes,  without 
losing  the  beneficial  features  of  MDA’s  existing  flexibility. 

In  the  near  term  we  recommend  that  the  Secretary  of  Defense  direct  MDA 
to  undertake  the  following  10  actions: 

In  the  area  of  cost: 

1.  Complete  total  cost  baselines  before  requesting  additional  funding 
for  Blocks  2.0  and  3.0  and  commit  to  a  date  when  baselines  for  all 
blocks  will  be  established. 

2.  Ensure  that  transfers  of  work  from  one  block  to  another  are 
transparent  and  reported  as  cost  variances. 


Page  60  GAO-09-338  Defense  Acquisitions 


Recommendations  for 
Executive  Action 


3.  Provide  additional  unit  costs  reports,  beyond  flyaway  unit  costs, 
that  incorporate  both  procurement  and  research  and  development 
funding  so  that  there  is  a  more  comprehensive  understanding  of 
the  progress  of  the  acquisitions. 

In  the  area  of  testing  and  performance: 

4.  Expand  the  BMDS  test  baseline  to  include  tests  scheduled  beyond 
the  first  succeeding  year  of  the  plan  to  ensure  its  synchronization 
with  BMDS  contracts. 

5.  Ensure  that  DOT&E  is  consulted  before  making  significant  changes 
to  the  test  baseline  so  that  the  tests  planned  provide  DOT&E  with 
sufficient  data  to  assess  the  performance  of  the  BMDS  elements. 

6.  Ensure  that  planned  test  objectives  include  concrete  data 
requirements  anchoring  models  and  simulations  to  real-world  tests, 
synchronized  with  flight  and  ground  test  plans  and  that  the  effects 
on  models  and  simulations  of  test  cancellations,  delays  or 
problems  are  clearly  identified  and  reported. 

7.  Reassess  the  flight  tests  scheduled  for  the  end  of  fiscal  year  2009  to 
ensure  that  they  can  be  reasonably  conducted  and  analyzed  given 
targets  and  other  constraints. 

In  the  area  of  knowledge-based  decisions: 

8.  Synchronize  the  development,  manufacturing,  and  fielding 
schedules  of  BMDS  assets  with  the  testing  and  validation 
schedules  to  ensure  that  items  are  not  manufactured  for  fielding 
before  their  performance  has  been  validated  through  testing. 

9.  Conduct  a  flight  test  of  the  CE-I  EKV  against  a  complex  target 
scene  with  countermeasures  to  complete  MDA’s  previous  testing 
goal  of  understanding  the  performance  capabilities  of  the  first  24 
fielded  GBIs. 

10.  Strengthen  the  capability  declarations  by  using  the  complete 
analysis  from  annual  performance  assessments  as  the  basis  for 
declaring  engagement  sequence  groups  as  fully  capable  and  block 
development  as  fully  complete;  otherwise,  indicate  the  limitations 
of  the  capabilities  and  steps  that  MDA  will  take  to  reduce  the  risks. 
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Agency  Comments 
and  Our  Evaluation 


DOD  provided  written  comments  on  a  draft  of  this  report.  These 
comments  are  reprinted  in  appendix  I.  DOD  also  provided  technical 
comments,  which  were  incorporated  as  appropriate. 

DOD  fully  concurred  with  10  of  our  11  recommendations  and  partially 
concurred  with  our  recommendation  that  the  Secretary  of  Defense  direct 
MDA  to  synchronize  the  development,  manufacturing,  and  fielding 
schedules  of  BMDS  assets  with  testing  and  validation  schedules  to  ensure 
that  items  are  not  manufactured  for  fielding  before  their  performance  has 
been  validated  through  testing.  Yet,  even  DOD’s  response  to  this 
recommendation  appears  to  be,  in  substance,  concurrence. 

DOD  concurred  with  our  recommendation  that  the  Secretary  of  Defense 
direct  MDA  to  ensure  that  transfers  of  work  from  one  block  to  another  are 
transparent  and  reported  as  cost  variances.  DOD  noted  in  its  response  that 
MDA  will  report  block  baselines  and  variances  annually  to  Congress  in  the 
BMDS  Accountability  Report.  The  Department  further  noted  that  for  the 
purposes  of  unit  cost  reporting,  MDA  has  defined  a  cost  variance  as  a 
confirmed  increase  of  10  percent  or  more  in  block  or  unit  costs  when 
compared  to  the  current  cost  baseline  or  20  percent  or  more  compared  to 
the  original  cost  baseline,  stating  that  transfers  of  work  creating  such  cost 
variances  will  be  reported.  The  intent  of  our  recommendation  is  to 
increase  visibility  into  transfers  of  work  between  blocks  regardless  of  the 
amount  of  the  increase  or  the  baseline  status  of  the  blocks.  The  trigger  for 
reporting  the  variances  selected  by  DOD  will  not  necessarily  provide  that 
visibility.  Given  that  only  between  2  and  26  percent  of  BMDS  block  and 
capability  development  costs  from  fiscal  year  2010  to  2013  will  be 
baselined  initially,  visibility  into  transfers  into  blocks  that  are  not  yet 
baselined  may  not  occur.  Further,  an  increase  may  not  be  reported  in  the 
baselined  block  from  which  work  is  transferred  because  the  transfer 
would  actually  yield  a  decrease  in  the  cost  of  the  baselined  block.  An 
increase  would  also  not  be  reported  in  the  receiving  block  if  that  block  is 
not  baselined  or  if  the  transfer  did  not  increase  costs  above  the  threshold. 
MDA  may  need  to  consider  a  different  approach  to  reporting  that  captures 
meaningful  transfers  of  work  into  and  out  of  blocks  regardless  of  whether 
any  of  the  blocks  are  baselined.  MDA  should  work  with  Congress  to 
determine  what  constitutes  a  meaningful  or  significant  cost  increase. 

DOD  also  concurred  with  our  recommendation  that  the  Secretary  of 
Defense  strengthen  the  capability  declarations  by  using  the  complete 
analysis  from  annual  performance  assessments.  In  responding  to  our 
recommendation,  DOD  noted  that  if  there  is  limited  performance 
assessment  data,  the  overall  capability  assessment  will  factor  in  the 
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knowledge  gained  from  ground  tests  and  flight  tests  against  the  identified 
risks.  While  we  recognize  that  MDA  is  not  always  able  to  complete  all  of 
its  planned  tests  in  a  given  time  period,  when  MDA  decides  to  change  the 
planned  basis  for  its  capability  declarations  to  a  different  or  reduced  set  of 
data  it  is  important  for  the  agency  to  clearly  report  the  limitations  that 
affect  the  capability  declaration  as  well  as  the  mitigation  steps  it  is  taking. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense  and  to  the 
Director,  MDA.  In  addition,  the  report  will  be  available  at  no  charge  on  the 
GAO  Web  site  at  http://www.gao.gov. 

If  you,  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841.  Contact  Points  for  our  offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  The  major  contributors  are  listed  in  appendix  VI. 


Paul  Francis 

Director,  Acquisition  and  Sourcing  Management 
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List  of  Congressional  Committees 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Daniel  K.  Inouye 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Ike  Skelton 
Chairman 

The  Honorable  John  M.  McHugh 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  John  P.  Murtha 
Chairman 

The  Honorable  C.W.  Bill  Young 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Comments  from  the  Department 
of  Defense 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION 

TECHNOLOGY 

AND  LOGISTICS  fJAR  “3  2009 

Mr.  Paul  Francis 

Director,  Acquisition  and  Sourcing  Management 
U.  S.  Government  Accountability  Office 
44 1  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Francis: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report, 
GAO-09-338,  “DEFENSE  ACQUISITIONS:  Production  and  Fielding  of  Missile 
Defense  Components  Continue  With  Less  Testing  and  Validation  Than  Planned,''  dated 
February  1 1,  2009  (GAO  Code  120744).  Detailed  comments  on  the  report 
recommendations  are  enclosed. 


The  DoD  concurs  with  ten  of  the  draft  report’s  recommendations  and  partially 
concurs  with  one.  The  rationale  for  our  position  is  included  in  the  enclosure.  1  submitted 
separately  a  list  of  technical  and  factual  errors  for  your  consideration. 

We  appreciate  the  opportunity  to  comment  on  the  draft  report.  My  point  of 
contact  for  this  effort  is  Mr.  David  Crim,  (703)  697-5385,  david.crim@osd.mil. 


Sincerely, 


David  G.  Ahem 
Director 

Portfolio  Systems  Acquisition 


Enclosure: 
As  stated 
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Appendix  I:  Comments  from  the  Department 
of  Defense 


GAO  DRAFT  REPORT  DATED  FEBRUARY  11,  2009 
GAO-09-338  (GAO  CODE  120744) 


“DEFENSE  ACQUISITIONS:  PRODUCTION  AND  FIELDING  OF  MISSILE 
DEFENSE  COMPONENTS  CONTINUE  WITH  LESS  TESTING  AND 
VALIDATION  THAN  PLANNED” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Missile  Defense  Executive  Board  to  assess  how  the  transparency  and  accountability  of  Missile 
Defense  Agency  (MDA’s)  acquisitions  can  be  strengthen  to  enhance  oversight,  such  as  by 
adopting  relevant  aspects  of  DoD's  normal  requirements,  acquisition  and  budgeting  processes, 
without  losing  the  beneficial  features  of  MDA’s  existing  flexibility 

DoD  RESPONSE:  Concur.  As  GAO  noted,  DoD  has  recently  enhanced  the  transparency, 
accountability,  and  oversight  of  the  missile  defense  program.  For  example,  the  Missile  Defense 
Executive  board  (MDEB)  has  played  an  increasingly  important  role  in  Ballistic  Missile  Defense 
System  (BMDS)  policy  and  programmatic  decisions.  Existing  flexibilities  and  the  associated 
integrated  decision  authority  for  requirements,  acquisition,  and  budget  have  facilitated  MDA’s 
efforts  to  provide  critical  capabilities  to  the  war  fighter.  However,  it’s  an  appropriate  time  to 
take  a  fresh  look.  Such  a  review  might  also  identify  where  the  flexibilities  and  integrated 
decision  authority  granted  MDA  could  be  applied  beneficially  to  other  DoD  programs. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
complete  total  cost  baselines  before  requesting  additional  funding  for  blocks  two  and  three  and 
commit  to  a  date  when  baselines  for  all  blocks  will  be  established. 

DoD  RESPONSE:  Concur.  MDA  intends  to  present  its  cost  baselines  for  Blocks  2.0  and  3. 1/3.2 
in  the  BMDS  Accountability  Report  (BAR)  accompanying  the  President’s  Budget  for  Fiscal  Year 
(FY)  2010.  As  for  Blocks  3.3  and  5.0,  assuming  events  unfold  as  expected,  we  intend  to  baseline 
their  costs  no  later  than  the  issuance  of  next  year’s  BAR.  MDA  intends  to  baseline  Block  4.0 
within  one  fiscal  year  of  reaching  agreements  with  the  Czech  and  Polish  governments  and 
obtaining  needed  Congressional  approvals. 

RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
ensure  that  transfers  of  work  from  one  block  to  another  are  transparent  and  reported  as  cost 
variances. 

DoD  RESPONSE:  Concur.  MDA  will  report  block  baselines  and  variances  annually  to 
Congress  in  the  BMDS  Accountability  Report.  For  the  purposes  of  unit  cost  reporting,  MDA  has 
defined  a  cost  variance  as  a  confirmed  increase  of  10  percent  or  more  in  block  or  unit  costs  when 
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Appendix  I:  Comments  from  the  Department 
of  Defense 


compared  to  the  current  cost  baseline  or  20  percent  or  more  compared  to  the  original  cost 
baseline.  Transfers  of  work  creating  such  cost  variances  will  be  reported. 

RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
provide  additional  unit  costs  reports,  beyond  flyaway  unit  costs,  that  incorporate  both 
procurement  and  research  and  development  funding  so  that  there  is  a  more  comprehensive 
understanding  of  the  progress  of  the  acquisitions. 

DoD  RESPONSE:  Concur.  MDA  will  provide  additional  unit  cost  reports  that  include 
development  and  integration  costs,  flyway  costs,  initial  spares,  and  support  items  (incorporating 
research  and  development  or  procurement  funds  as  appropriated)  for  major  pieces  of  equipment 
such  as  interceptors,  sensors,  fire  control/command  systems,  and  launch  systems  in  baselined 
blocks. 

RECOMMENDATION  S:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
expand  the  Ballistic  Missile  Defense  Systems  (BMDS)  test  baseline  to  include  tests  scheduled 
beyond  the  first  succeeding  year  of  the  plan  to  ensure  its  synchronization  with  BMDS  contract. 

DoD  RESPONSE:  Concur.  The  Missile  Defense  Agency  is  developing  the  Integrated  Master 
Test  Plan  (IMTP)  that  spans  the  Future  Year  Defense  Program  (FYDP)  rather  than  in  two  year 
increments.  It  will  include  test  objectives  based  on  specifications,  modeling  and  simulation 
verification,  validation  and  accreditation  and  Critical  Operational  Issues. 

RECOMMENDATION  6:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
ensure  Director,  Operational  Test  and  Evaluation  (DOT&E)  is  consulted  before  making 
significant  changes  to  the  test  baseline  so  that  the  tests  planned  provide  the  Director  of  DOT&E 
with  sufficient  data  to  assess  the  performance  of  the  BMDS  elements. 

DoD  RESPONSE:  Concur.  Proposed  updates  to  the  test  baseline  and  the  IMTP  will  be 
coordinated  through  established  Working  Groups,  which  include  DOT&E,  Operational  Test 
Agencies  (OTAs),  and  the  War  fighter  as  permanent  members.  The  updates  will  continue  to  be 
staffed  through  MDA  Leadership  and  both  DOT&E  and  the  OTAs,  and  DOT&E  will  continue  as 
a  signator  to  the  IMTP. 

RECOMMENDATION  7:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
ensure  that  planned  test  objectives  include  concrete  data  requirements  anchoring  models  and 
simulations  to  real-world  tests,  synchronized  with  flight  and  ground  test  plans  and  that  the  effects 
on  models  and  simulations  of  test  cancellations,  delays  or  problems  are  clearly  identified  and 
reported. 

DoD  RESPONSE:  Concur.  Flight  tests  include  objectives  to  anchor  models  and  simulations  to 
real-world  test  data.  MDA  currently  does  a  post-flight-test  reconstruction  of  each  flight  test, 
which  includes  the  requirement  to  "re-fly"  flight  tests  in  our  ground  Hardware  in  Loop  Tests. 

This  helps  to  ensure  that  models  and  simulations  accurately  represent  element  and  system 
performance.  Additionally,  MDA  has  clearly  identified  and  reported  the  effects  on  models  and 
simulations  of  test  cancellations,  delays  or  problems  and  will  continue  to  do  so. 
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RECOMMENDATION  8:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
reassess  the  flight  tests  scheduled  for  the  end  of  fiscal  year  2009  to  ensure  that  they  can  be 
reasonably  conducted  and  analyzed  given  targets  and  other  constraints. 

DoD  RESPONSE:  Concur.  MDA  concurs  with  reassessment  of  the  test  schedule  as  it  applies  to 
the  conduct  of  flight  tests  through  the  end  of  fiscal  year  2009. 

RECOMMENDATION  9:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
synchronize  the  development,  manufacturing,  and  fielding  schedules  of  BMDS  assets  with 
testing  and  validation  schedules  to  ensure  that  items  are  not  manufactured  for  fielding  before 
their  performance  has  been  validated  through  testing. 

DoD  RESPONSE:  Partially  Concur.  MDA  is  pursuing  synchronization  of  development, 
manufacturing  and  fielding  of  BMDS  assets  with  the  lMTP's  testing  and  validation  requirements. 
That  synchronization  will  be  captured  in  the  draft  BMDS  Master  Plan  (BMP)  and  its  associated 
Integrated  Master  Plan  (IMP)  and  Integrated  Master  Schedule  (IMS).  While  successful  ground 
and  flight  tests  have  provided  confidence  in  BMDS  capabilities  being  fielded,  MDA  and  the  war 
fighter  recognize  that  additional  validation  through  modeling  and  simulation  is  needed. 

RECOMMENDATION  10:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
conduct  a  flight  test  of  the  Capability  Enhancement  (CE)-I  Exoatmospheric  Kill  Vehicle  (EKV) 
against  a  complex  target  scene  with  countermeasures  to  complete  MDA’s  previous  testing  goal 
of  understanding  the  performance  capabilities  of  the  first  24  fielded  Ground-Based  Interceptors 
(GBIs). 

DoD  RESPONSE:  Concur.  MDA  is  currently  reexamining  its  flight  testing  program  and 
expects  to  include  additional  flight  testing  of  the  Capability  Enhancement  CE-I  EKV.  This 
testing  will  be  reflected  in  the  next  IMTP  update.  Such  testing  will  include  the  specific  objective 
to  discriminate  and  intercept  the  dynamic  lethal  object  from  a  target  scene  with  countermeasures. 

RECOMMENDATION  11:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  MDA  to 
strengthen  the  capability  declarations  by  using  the  complete  analysis  from  annual  performance 
assessments  as  the  basis  for  declaring  engagement  sequence  groups  as  fully  capable  and  block 
development  as  fully  complete;  otherwise,  indicate  the  limitations  of  the  capabilities  and  steps 
the  MDA  will  take  to  reduce  the  risks. 

DoD  RESPONSE:  Concur.  MDA  makes  a  capability  declaration  based  on  complete  analysis  of 
data  from  all  available  ground  test,  flight  test  and  performance  assessment  events.  As  part  of  the 
technical  assessment  criteria  for  a  capability  delivery  (whether  an  engagement  sequence  group  or 
BMDS  Block  capability),  MDA  identifies  the  capabilities  and  limitations  and  provides  the  MDA 
Director  with  the  summary  of  any  remaining  risk  to  the  capability.  If  there  is  limited 
performance  assessment  data,  the  overall  capability  assessment  will  factor  in  the  knowledge 
gained  from  ground  tests  and  flight  tests  against  the  identified  risks. 
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Based  on  our  analysis  of  14  Ballistic  Missile  Defense  System  (BMDS) 
elements’  prime  contractor  earned  value  management  performance,  we 
determined  that  collectively  the  contractors  overran  budgeted  cost  by 
$152.4  million  and  were  behind  schedule  by  approximately  $107.4  million 
during  the  fiscal  year.1  Our  insight  of  the  dollar  gained  or  lost  for  each 
dollar  invested  is  based  on  monthly  earned  value  reports  which  are 
required  of  each  BMDS  program  office’s  prime  contractor.  These  reports 
compare  monthly  progress  to  the  cost  or  schedule  performance  baseline 
to  reveal  whether  the  work  scheduled  is  being  completed  on  time  and  if 
the  work  is  being  completed  at  the  cost  budgeted.  For  example,  if  the 
contractor  was  able  to  complete  more  work  than  scheduled  and  for  less 
cost  than  budgeted,  the  contractor  reports  a  positive  schedule  and  cost 
variance.  Alternatively,  if  the  contractor  was  not  able  to  complete  the 
work  in  the  scheduled  time  period  and  spent  more  than  budgeted,  the 
contractor  reports  both  a  negative  schedule  and  cost  variance.  The  results 
can  also  be  mixed  by,  for  example,  completing  the  work  under  cost  (a 
positive  cost  variance)  but  taking  longer  than  scheduled  to  do  so  (a 
negative  schedule  variance). 

We  also  used  contract  performance  report  data  to  base  predictions  of 
likely  overrun  or  underrun  of  each  prime  contractor’s  budgeted  cost  at 
completion.  Our  predictions  of  final  contract  cost  are  based  on  the 
assumption  that  the  contractor  will  continue  to  perform  in  the  future  as  it 
has  in  the  past.  In  addition,  since  they  provide  the  basis  for  our  projected 
overruns,  we  also  provide  the  total  budgeted  contract  cost  at  completion 
for  each  contract  we  assessed  in  this  appendix.2  However,  the  budgeted 
costs  at  completion,  in  some  cases,  have  grown  significantly  over  time.  For 
example,  in  one  case  the  budgeted  cost  at  completion  increased  by 
approximately  five  times  its  original  value.  Since  our  assessment  does  not 
reveal,  as  cost  growth,  the  difference  between  the  original  and  current 
budgeted  costs  at  completion  it  would  be  inappropriate  to  compare  the 
underruns  or  overruns  for  MDA  programs  with  cost  growth  on  major 


1  Earned  Value  Management  is  a  program  management  tool  that  integrates  the  technical, 
cost,  and  schedule  parameters  of  a  contract.  During  the  planning  phase,  an  integrated 
baseline  is  developed  by  time  phasing  budget  resources  for  defined  work.  As  work  is 
performed  and  measured  against  the  baseline,  the  corresponding  budget  value  is  “earned.” 
Using  this  earned  value  metric,  cost  and  schedule  variances  can  be  determined  and 
analyzed. 

2  The  total  contract  cost  at  completion  is  based  on  budgeted  cost  at  completion  for  each 
contract  we  assessed.  The  budget  at  completion  represents  the  total  planned  value  of  the 
contract. 
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defense  acquisition  programs  since  those  major  defense  acquisition 
programs  have  established  their  full  scope  of  work  as  well  as  developed 
total  cost  baselines,  while  these  have  not  been  developed  for  MDA 
programs.3 


Aegis  BMD 
Contractors 
Experienced  Mixed 
Performance  during 
the  Fiscal  Year 


The  Aegis  Ballistic  Missile  Defense  (Aegis  BMD)  program  manages  two 
prime  contractors  for  work  on  its  two  main  components — the  Aegis  BMD 
Weapon  System  and  the  Standard  Missile-3  (SM-3).  We  report  on  work 
under  one  of  the  two  separate  Aegis  BMD  SM-3  contract’s  contract  line 
item  numbers  (CLIN)on  which  we  received  sufficient  performance  data 
during  fiscal  year  2008.  The  first  Aegis  BMD  SM-3  contract’s  CLIN  9  was 
for  the  production  of  20  Block  1A  missiles  which  began  in  February  2007 
and  finished  deliveries  in  August  2008.  Deliveries  were  completed  $7.5 
million  under  budget  on  the  contractor’s  total  budgeted  cost  of  $179.0 
million.  The  other  Aegis  BMD  SM-3  contract’s  CLIN  1  is  for  a  fourth  lot  of 
27  Block  1A  missiles  and  began  reporting  performance  data  in  August  2007 
for  work  that  is  still  ongoing.  The  weapon  system  contractor  experienced 
cost  growth  and  schedule  delays  while  the  SM-3  contractor  for  the  ongoing 
CLIN  1  for  27  Block  1A  missiles  had  mixed  performance.  Neither  of  these 
CLINs  experienced  a  realignment  during  fiscal  year  2008. 


The  Aegis  BMD  weapon  system  contractor  experienced  cumulative  cost 
growth  and  schedule  delays  throughout  the  year.  The  Aegis  BMD  weapon 
system  contractor  overran  budgeted  cost  and  schedule  during  the  fiscal 
year  by  $7  million  and  $5.1  million  respectively.  Although  cumulative  cost 
performance  remains  positive  at  $16  thousand,  cumulative  schedule 
performance  continued  to  decline  to  negative  $8.4  million.  The  negative 
cumulative  schedule  variance  is  driven  by  late  engineering  data,  delays  to 
qualification  efforts,  and  the  need  to  return  components  experiencing 
issues  back  to  the  vendor  which  required  more  time  than  originally 
planned.  See  figure  7  for  cumulative  cost  and  schedule  performance 
during  the  fiscal  year. 


3  The  current  budgeted  costs  at  completion  are  as-of  September  30,  2008. 
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Figure  7:  Aegis  BMD  Weapon  System  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Cumulative  schedule  variance 
Source:  Contractor  (data),  GAO  (presentation). 


The  program  attributes  the  fiscal  year  cost  and  schedule  overruns  mainly 
to  the  additional  time  and  testing  needed  to  ensure  that  the  weapon 
system  fielded  capability  was  what  was  originally  promised  to  the 
warfighter.  To  account  for  some  of  the  overruns,  the  program  performed 
fewer  risk  reduction  efforts  for  a  future  weapon  system  capability  release. 
If  the  contractor  continues  to  perform  as  it  has  during  the  fiscal  year,  we 
project  that  at  contract  completion  in  September  2010,  the  contractor  will 
overrun  its  budgeted  cost  of  $1.2  billion  by  between  $1.9  million  and  $12.2 
million. 

The  Aegis  BMD  SM-3  contractor,  producing  another  lot  of  27  Block  1A 
missiles  under  its  CLIN  1,  ended  the  fiscal  year  by  underrunning  budgeted 
costs  by  $3.0  million.  The  Aegis  BMD  SM-3  contractor  for  CLIN  1  work 
also  ended  the  year  with  a  negative  $7.6  million  schedule  variance,  which 
means  that  the  contractor  was  unable  to  accomplish  $7.6  million  worth  of 
planned  work.  Since  reporting  began  in  August  2007,  cumulative  and  fiscal 
year  variances  are  nearly  equal  with  cumulative  cost  variances  at  a 
positive  $3.3  million  and  cumulative  schedule  variances  at  negative  $7.0 
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million.  See  figure  8  for  a  graphic  representation  of  the  cumulative  cost 
and  schedule  variances  during  fiscal  year  2008. 


Figure  8:  Aegis  BMD  SM-3  CLIN  1  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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hh  Cumulative  cost  variance 

Cumulative  schedule  variance 
Source:  Contractor  (data),  GAO  (presentation). 


The  contractor  was  able  to  accomplish  fiscal  year  2008  work  for  $3.0 
million  less  than  originally  planned  in  part  due  to  adjustments  made  in 
program  management,  labor  efficiencies,  reductions  in  vendor  cost,  and 
material  transfers  in  the  missile’s  fourth  stage  component.  The 
unaccomplished  work  in  negative  $7.6  million  worth  of  fiscal  year 
schedule  variances  is  largely  in  the  first,  second,  and  fourth  stages  portion 
of  work.  In  the  first  stage  booster,  the  contractor  attributes  some  of  the 
negative  schedule  variance  to  more  than  a  year  delay  in  testing  the  first 
stage  due  to  rework  needed  to  correct  errors  in  the  original  drawing 
packages.  In  addition,  the  contractor  cites  second  stage  component 
delivery  delays  as  drivers  for  the  negative  schedule  variance.  Vendors 
were  unable  to  deliver  these  components  due  to  holdups  in  approving 
waivers,  achieving  recertification  after  test  equipment  failures,  and 
property  damage  to  facilities.  Lastly,  the  contractor  experienced  delays  in 
components  for  the  fourth  stage  which  also  contributed  to  the  unfavorable 
schedule  variance.  If  the  contractor  continues  to  perform  as  it  did  through 
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ABL  Contractor 
Overran  Budgeted 
Fiscal  Year  Cost 


September  2008,  our  analysis  predicts  that,  at  completion  in  April  2010,  the 
work  under  the  contract  could  cost  from  $6.6  million  less  to  $0.7  million 
more  than  the  budgeted  cost  of  $237.5  million. 


For  fiscal  year  2008,  the  Airborne  Laser  (ABL)  contractor  overran  fiscal 
year  budgeted  costs  by  $10.6  million  but  had  a  positive  fiscal  year  schedule 
variance  of  $2.2  million.  Despite  some  gains  in  its  schedule  variance  during 
the  fiscal  year,  the  program  still  maintains  negative  cumulative  cost  and 
schedule  variances  of  $84.8  million  and  $23.6  million  respectively.  The 
contractor  mostly  attributes  the  negative  cumulative  variances  in  cost  and 
schedule  to  late  beam  control/fire  control  hardware  deliveries.  Despite  a 
replan  in  June  2007,  the  ABL  contractor  did  not  perform  any  type  of 
realignment  during  fiscal  year  2008.  Figure  9  shows  cumulative  variances 
at  the  beginning  of  fiscal  year  2008  along  with  a  depiction  of  the 
contractor’s  cost  and  schedule  performance  throughout  the  fiscal  year. 


Figure  9:  ABL  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation). 


Technical  issues  with  key  components  of  the  Beam  Control  Fire  Control 
system  that  required  new  hardware  or  refurbishment  of  existing 
components  as  well  as  late  deliveries  of  key  laser  system  components  are 
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the  primary  drivers  of  the  unfavorable  fiscal  year  cost  variance  of  $10.6 
million.  These  issues  have  caused  delays  in  integration  and  test  activities 
for  the  overall  ABL  weapon  system.  Based  on  the  contractor’s 
performance  up  through  fiscal  year  2008,  we  estimate  that,  at  completion 
in  February  2010,  the  contractor  will  overrun  its  budgeted  cost  of  $3.6 
billion  by  between  $89.7  million  and  $95.4  million. 


C2BMC  Program 
Incurred  Negative 
Cumulative  and  Fiscal 
Year  Variances 


Our  analysis  of  the  Command,  Control,  Battle  Management,  and 
Communications’  (C2BMC)  cumulative  contract  performance  indicates 
that  the  prime  contractor’s  performance  declined  during  fiscal  year  2008. 
The  contractor  overran  its  fiscal  year  2008  budget  by  $9.8  million  and  did 
not  perform  $3.6  million  of  work  on  schedule.  By  September  2008,  this 
resulted  in  an  unfavorable  cumulative  cost  variance  of  $24.3  million  and  an 
unfavorable  cumulative  schedule  variance  of  $7.1  million.  The  main 
drivers  for  the  negative  cumulative  cost  variances  were  costs  associated 
with  unplanned  work,  increased  technical  complexity,  and  reduction  to 
cost  efficiency  due  to  losing  key  staff.  The  contractor  attributes  the 
unfavorable  cumulative  schedule  variances  to  software  issues  related  to 
the  global  engagement  manager  and  components  of  test  training 
operations.  Although  the  C2BMC  contractor  performed  a  replan  in 
November  2006,  the  contractor  did  not  perform  any  type  of  realignment 
during  fiscal  year  2008.  Trends  in  cost  and  schedule  performance  during 
the  fiscal  year  are  depicted  in  figure  10. 
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Figure  10:  C2BMC  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation). 

The  negative  fiscal  year  cost  variance  of  $9.8  million  is  driven  mainly  by 
problems  in  the  performance  of  work  under  Part  4  and  Part  5  of  the 
contract.  The  Part  4  effort,  which  began  in  December  2005,  includes 
completing  several  spiral  capabilities,  upgrading  spiral  suites,  and 
implementing  initial  global  engagement  capabilities  at  an  operations 
center.  The  Part  5  effort  began  in  December  2007  and  covers  operations 
and  sustainment  support  for  fielded  C2BMC;  the  delivery  of  spiral 
hardware,  software,  and  communications;  as  well  as  development, 
planning,  and  testing  for  other  spiral  capabilities.  The  contractor  was  able 
to  use  reserves  to  cover  some  of  its  Part  4  unfavorable  fiscal  year  cost 
variances. 

The  Part  5  fiscal  year  cost  variance’s  primary  drivers  are  unexpected 
complexities  with  the  network  design,  unplanned  work  that  required  more 
resources  for  developing  the  planner,  and  the  extension  of  efforts  past  the 
completion  date  on  the  global  engagement  management  portion  of  work. 
The  unfavorable  fiscal  year  schedule  variance  of  $3.6  million  is 
attributable  to  the  Part  5  portion  of  work  and  primarily  caused  by  an 
unexpected  reallocation  of  resources  off  of  the  global  engagement 
management  portion  of  work  to  other  areas,  delays  in  requesting  material 


Page  75 


GAO-09-338  Defense  Acquisitions 


Appendix  II:  BMDS  Prime  Contractors  Exceed 
Budgeted  Cost  and  Schedule  Performance 
during  Fiscal  Year  2008 


procurement  also  for  global  engagement  management,  and  a  lagging 
schedule  for  building  out  a  testing  lab.  If  the  contractor  continues  to 
perform  as  it  has  in  the  past,  we  predict  that  the  contractor  will  overrun  its 
budgeted  cost  of  $1.0  billion  at  completion  in  December  2009  by  between 
$37.1  million  and  $76.8  million. 


GMD  Contractor 
Maintained  Negative 
Cumulative  Cost  and 
Schedule  Variances 
throughout  the  Fiscal 
Year 


The  government  and  the  Ground-based  Midcourse  Defense  (GMD) 
contractor  began  a  contract  restructuring  during  the  fiscal  year  to  rephase 
and  rescope  on-going  efforts  to  refine  capability  requirements  and  to 
adjust  program  content  as  well  as  perform  weapon  system  integration, 
perform  flight  test  planning,  and  work  to  develop  the  two-stage  booster 
among  other  tasks.  The  ongoing  realignment  includes  a  proposal  to  add 
between  $360  million  and  $680  million  to  the  cost  of  the  work  under 
contract  and  to  add  39  months  to  the  period  of  performance. 

The  GMD  contractor  reports  a  cumulative  negative  cost  variance  of  more 
than  $1.0  billion  that  it  attributes  to  technical  challenges  with  its 
Exoatmospheric  Kill  Vehicle  (EKV)  as  well  as  supplier  component  quality 
problems.  The  contractor  also  carries  a  total  unfavorable  cumulative 
schedule  variance  of  $130.3  million,  the  bulk  of  which  the  contractor 
attributes  to  the  technical  issues  connected  with  the  Ground-based 
Interceptor  (GBI),  particularly  the  EKV.  For  example,  during  the  fiscal 
year  the  program  experienced  difficulties  in  manufacturing  the  Capability 
Enhancement  II  (CE-II)  EKVs.  Although  the  CE-II  EKVs  are  expected  to 
provide  better  performance,  the  contractor  produced  the  kill  vehicles 
before  completing  developmental  tests,  discovered  problems  during 
manufacturing,  incorporated  a  new  design,  and  continued  manufacturing 
them.  Although  these  issues  contributed  unfavorable  fiscal  year  cost 
variances  of  $42.7  million,  the  program  was  able  to  make  up  for  these 
losses  in  other  areas.  The  variances,  depicted  in  figure  11  represent  the 
GMD  contractor’s  cumulative  cost  and  schedule  performance  over  fiscal 
year  2008. 
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Figure  1 1 :  GMD  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation). 


The  GMD  contractor  did  have  a  favorable  fiscal  year  cost  variance  of  $53.9 
million,  which  it  attributed  to  labor  efficiencies  in  the  ground  system  as 
well  as  less  field  maintenance  support  required  than  planned,  and  labor 
efficiencies  in  the  deployment  and  sustainment  portion  of  the  work  under 
the  contract.  However,  the  GMD  element’s  underruns  occurred  partially 
because  the  contractors  delayed  or  eliminated  some  planned  work.  For 
example,  the  GMD  program  did  not  accomplish  the  emplacement  of  three 
GBIs,  or  conduct  either  of  its  two  planned  flight  tests.  As  a  result,  it 
employed  less  labor  than  originally  intended.  The  program  also  reports  an 
unfavorable  fiscal  year  schedule  variance  of  $77.4  million  which  it 
attributes  to  an  administrative  error  that  occurred  in  September  2007.  This 
error  incorrectly  adjusted  the  baseline  to  the  booster  effort  in  September 
which  was  then  updated  in  October.  However,  it  should  also  be  noted  that 
Missile  Defense  Agency  (MDA)  officials  believe  that  ongoing  adjustments 
to  the  GMD  element’s  baseline  have  skewed  recent  variances  to  such  a 
degree  that  they  should  not  be  used  to  predict  future  costs.  We  did 
perform  analysis  based  on  the  contractor’s  reported  performance  through 
fiscal  year  2008,  and  our  analysis  estimates  that  at  contract  end  planned 
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for  December  2011,  the  contractor  could  overrun  its  budgeted  cost  of 
$14.9  billion  by  between  $950.2  million  and  $1.25  billion. 


KEI  Cost  and 
Schedule 
Performance 
Continued  to  Decline 
after  Replan 


Despite  a  replan  in  April  2007  and  again  in  April  2008,  the  Kinetic  Energy 
Interceptors  (KEI)  contractor  continued  to  experience  declining  cost  and 
schedule  performance  during  the  fiscal  year.  Although  the  contractor 
began  the  year  with  a  positive  cost  variance,  the  contractor  overran  fiscal 
year  2008  budgeted  costs  by  $8.3  million,  ending  the  year  with  an 
unfavorable  cumulative  cost  variance  of  $2.6  million.  In  addition,  the 
program  was  unable  to  accomplish  $8.5  million  worth  of  work  which 
added  to  an  unfavorable  cumulative  schedule  variance  of  $21.3  million. 
Cumulative  cost  and  schedule  variances  were  mainly  driven  by  costs 
associated  with  delays  to  booster  drawing  releases,  delays  in  procurement, 
and  unexpected  costs  and  rework  related  to  issues  with  the  second  stage. 
Figure  12  depicts  the  cost  and  schedule  performance  for  the  KEI 
contractor  during  fiscal  year  2008. 


Figure  12:  KEI  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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The  KEI  contractor  attributes  its  unfavorable  fiscal  year  cost  and  schedule 
variances  of  $8.3  million  and  $8.5  million,  respectively,  to  issues  with  its 
interceptor  booster.  Problems  initially  arose  in  fiscal  year  2007  with  a 
motor  case  failure  during  acceptance  testing  which  led  to  unexpected 
redesigns.  In  October  2007,  the  program  experienced  several  issues  with 
the  nozzle  during  a  second  stage  ground  test  and  also  experienced  a 
deviation  in  measured  performance  from  pre-test  predictions.  These  issues 
added  costly  redesigns  and  delays  to  its  knowledge  point,  a  booster  flight 
test.  The  program  performed  a  replan  of  its  work  in  April  2008  because  of 
these  issues  in  order  to  realign  the  schedule  with  their  booster  flight  test 
knowledge  point  which  was  delayed  from  August  2008  to  April  2009.  Since 
the  replan,  the  booster  flight  test  has  been  further  delayed  to  the  fourth 
quarter  of  fiscal  year  2009.  As  a  result  of  the  replan,  the  program  zeroed 
out  some  schedule  variances  from  the  baseline  to  reflect  the  program’s 
progress  toward  the  newly  defined  schedule.  Despite  this  replan  in  April, 
our  analysis  shows  that  the  replan  has  not  improved  overall  performance 
as  cumulative  cost  and  schedule  variances  continue  their  downward  trend. 
We  were  unable  to  estimate  whether  the  total  work  under  the  contract  is 
likely  to  be  completed  within  budgeted  cost  since  trends  cannot  be 
developed  until  at  least  15  percent  of  the  work  under  the  contract  is 
completed. 


Limited  Contractor 
Data  Prevented 
Analysis  of  All  MKV 
Task  Orders 


The  Multiple  Kill  Vehicles  (MKV)  program  began  utilizing  an  indefinite 
delivery  indefinite  quantity  contract  in  January  2004.  Since  then,  the 
program  has  initiated  eight  task  orders,  five  of  which  were  open  during 
fiscal  year  2008 — Task  Orders  4  through  8.  Task  Order  4  provided 
insufficient  data  to  complete  full  earned  value  analysis  for  the  fiscal  year. 
In  addition,  Task  Order  5  was  completed  shortly  after  the  fiscal  year 
began,  without  providing  enough  data  to  show  performance  trends. 
Therefore  we  performed  analysis  for  Task  Orders  6,  7,  and  8  as  shown 
below.  None  of  the  task  orders  were  realigned  during  the  fiscal  year. 

MKV  Task  Order  6  began  in  November  2006  for  the  component 
development  and  testing  of  a  prototype  carrier  vehicle  seeker  (a  long- 
range  sensor).  According  to  the  task  order,  this  seeker  for  the  carrier 
vehicle  will  assign  individual  kill  vehicles  for  target  destruction.  This  task 
will  culminate  in  a  demonstration  planned  for  fiscal  year  2010.  As  shown 
in  figure  13  below,  performance  data  over  the  course  of  the  fiscal  year 
illustrates  declining  cost  and  schedule  performance.  Although  it  began  the 
fiscal  year  with  slightly  positive  cumulative  cost  and  schedule  variances, 
the  program  ended  the  year  with  slightly  negative  cumulative  cost  and 
schedule  variances  of  $1.1  million  and  $0.6  million  respectively.  In 
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addition,  the  contractor  has  unfavorable  fiscal  year  cost  and  schedule 
variances  of  $1.4  million  and  $1.5  million,  respectively.  The  program 
attributes  its  negative  cumulative  cost  and  schedule  variances  to  increased 
work  necessary  to  resolve  software  development  issues,  unplanned  efforts 
as  a  result  of  late  hardware  arrivals,  and  a  government-directed  change  in 
vendors  for  hardware  resulting  in  additional  design  work.  Based  on  our 
analysis  and  the  assumption  that  the  contractor  will  continue  to  perform 
as  it  has  through  fiscal  year  2008,  we  predict  that  at  its  contract 
completion  in  May  2009,  the  contractor  on  Task  Order  6  will  overrun  its 
budgeted  cost  of  $19.3  million  by  between  $1.6  million  and  $2.5  million. 


Figure  13:  MKV  Task  Order  6  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation) 


The  MKV  Task  Order  7  is  for  the  development  and  testing  of  engagement 
management  algorithms  and  the  test  bed  in  which  it  will  be  demonstrated. 
These  algorithms  will  be  a  critical  capability  of  the  carrier  vehicle  to 
manage  the  kill  vehicle  engagements  relying  on  target  information  from 
the  BMDS  sensors  and  the  carrier  vehicle  long-range  sensor.  The 
contractor  on  this  task  order  performed  positively  during  the  fiscal  year, 
both  in  terms  of  its  cumulative  and  fiscal  year  cost  and  schedule  variances. 
The  program  had  a  favorable  fiscal  year  cost  variance  of  $1.4  million  and  a 
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positive  fiscal  year  schedule  variance  of  $11  thousand,  adding  to  its 
favorable  cumulative  cost  and  schedule  variances  of  $1.7  million  and  $0.1 
million,  respectively.  The  program  attributes  its  cumulative  cost  underruns 
to  several  reasons  including  a  programmatic  decision  to  proceed  with  one 
approach  for  organizing  kill  vehicles  in  attack  formation  rather  than 
funding  several  different  approaches.  In  addition,  the  contractor 
experienced  cost  savings  with  greater  efficiencies  than  expected  in  the  kill 
vehicle  portion  of  the  work  under  the  contract  and  less  manpower  than 
planned  in  other  portions  of  the  work  under  the  contract.  If  the  contractor 
continues  to  perform  as  it  has  in  the  past,  we  estimate  that  at  completion 
in  May  2010  the  work  under  the  contract  could  cost  between  $3.2  million 
and  $3.9  million  less  than  the  expected  $43.9  million  budgeted  for  the 
work  under  the  contract.  See  figure  14  below  for  an  illustration  of 
cumulative  cost  and  schedule  performance  during  fiscal  year  2008. 


Figure  14:  MKV  Task  Order  7  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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MKV  Task  Order  8  was  awarded  in  January  2007  and  began  reporting  full 
performance  data  in  July  2007.  The  task  order  is  for  the  development  and 
testing  of  a  hover  test  bed  and  hover  test  vehicle.  This  hover  test  bed  will 
allow  the  program  to  integrate  and  test  key  components  of  the  system  in  a 
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repeatable  ground-based  free  flight  environment  as  their  technologies 
reach  maturity.  The  program  experienced  a  continuing  schedule 
performance  decline  as  seen  in  figure  15. 


Figure  15:  MKV  Task  Order  8  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation). 


Although  the  contractor  began  the  year  with  a  positive  cumulative  cost 
variance,  overruns  during  the  fiscal  year  of  $10.7  million  led  the  program 
to  a  total  cumulative  cost  overrun  of  $10.3  million.  The  element’s  fiscal 
year  schedule  variance  was  slightly  negative  at  an  unfavorable  $15 
thousand,  leaving  its  cumulative  schedule  variance  largely  unchanged  at  a 
favorable  $0.3  million.  The  program  attributes  the  cumulative  cost 
variances  to  increased  labor,  procurement,  and  material  costs  as  well  as 
increased  hardware  and  engineering  drawings,  and  management  oversight 
to  resolve  subcontractor  inefficiencies.  In  addition,  the  program  increased 
expenditures  to  resolve  technical  and  schedule  issues  associated  with  the 
development  of  avionics  subsystems.  The  planned  date  for  the  task  order’s 
main  effort — completing  the  hover  test — was  delayed  2  months  from  its 
original  date  to  December  2008  in  part  because  of  technical  issues 
associated  with  the  test  vehicle’s  power  unit  and  a  software  anomaly. 
These  issues  were  resolved  prior  to  the  hover  test  being  conducted.  Based 
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on  its  prior  performance,  the  MKV  contractor  could  overrun  the  budgeted 
cost  of  $48.0  million  for  the  work  under  the  contract  at  completion  in 
January  2009  by  between  $5.7  million  and  $13.8  million. 


Sensors’  Radar 
Experienced  Fiscal 
Year  Cost  and 
Schedule  Growth 


As  of  September  2008,  the  Sensor’s  contractor  had  overrun  its  fiscal  year 
budget  by  $2.2  million  and  was  behind  in  completing  $27.4  million  worth 
of  work.  Considering  prior  years’  performance,  the  contractor  is 
performing  under  budget  with  a  favorable  cumulative  cost  variance  of 
$22.0  million.  However,  the  contractor  has  a  cumulative  unfavorable 
schedule  variance  of  $9.6  million.  The  contractor  reports  the  cumulative 
schedule  variance  is  driven  by  delays  in  the  manufacturing  of  the  sixth 
radar  and  a  software  capability  release  that  is  2  to  3  months  behind 
schedule.  Additionally,  the  contractor  reports  that  its  favorable  cumulative 
cost  variance  is  attributable  to  efficiencies  in  the  second  radar’s 
manufacturing,  design,  development,  and  software.  The  Sensors 
contractor  has  not  performed  a  realignment  of  its  work  since  contract 
start  in  April  2003.  See  figure  16  for  trends  in  the  contractor’s  cost  and 
schedule  performance  during  the  fiscal  year. 


Figure  16:  Sensors  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation). 
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The  contractor  reports  that  its  unfavorable  fiscal  year  schedule  variance  of 
$27.4  million  is  due  to  a  decrease  of  previously  earned  positive  schedule 
variances  reaped  from  the  manufacturing  efficiencies  leveraged  from  the 
Terminal  High  Altitude  Area  Defense  radar  hardware  design.  In  addition, 
late  delivery  of  components  also  contributed  to  the  negative  fiscal  year 
schedule  variances.  The  negative  fiscal  year  cost  variance  of  $2.2  million  is 
largely  due  to  a  contract  change  related  to  its  incentive  fee.  Our  analysis 
predicts  that  if  the  contractor  continues  to  perform  as  it  has  through  fiscal 
year  2008,  the  work  under  the  contract  could  cost  from  $25  million  less  to 
$9.1  million  more  than  the  budgeted  cost  of  $1.1  billion  at  completion 
currently  planned  for  December  2010. 


Technical  Issues 
Drove  STSS  Cost 
Growth  during  the 
Fiscal  Year 


After  a  replan  of  work  in  October  2007,  the  Space  Tracking  and 
Surveillance  System  (STSS)  contractor  experienced  an  unfavorable  cost 
variance  of  $87.9  million  during  the  fiscal  year.  The  replan  was  undertaken 
in  order  to  extend  the  period  of  performance  and  delay  the  launch  date  of 
its  demonstrator  satellite.  Despite  fiscal  year  cost  overruns,  the  contractor 
was  able  to  make  gains  on  the  cumulative  schedule  variance  by 
accomplishing  $1.9  million  more  worth  of  work  than  was  originally 
planned.  Cumulatively,  the  program  has  both  unfavorable  cost  and 
schedule  variances  at  $319.3  million  and  $17.8  million,  respectively.  The 
program  attributes  cumulative  cost  variances  and  schedule  variances  to 
continual  launch  date  schedule  slippages.  In  addition,  problems  in  the 
space  segment  portion  of  work  also  added  to  the  cumulative  cost 
variances.  Figure  17  shows  both  cost  and  schedule  trends  during  fiscal 
year  2008. 
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Figure  17:  STSS  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Program  cost  variances  during  the  fiscal  year  were  driven  primarily  by 
technical  issues  with  hardware  installed  on  the  second  space  vehicle. 
These  issues  included  an  overheating  flight  communications  box,  a  leak  on 
the  propulsion  side  of  the  satellite,  and  problems  with  the  spacecraft 
processor  that  failed  to  send  a  critical  command  to  the  onboard  computer. 
To  resolve  the  issues  with  the  processor,  the  program  office  initially 
recommended  the  removal  of  the  entire  computer  from  the  spacecraft. 
However,  after  extensive  research  and  testing,  the  program  manager 
determined  that  the  event  with  the  spacecraft  is  an  unverifiable  failure 
with  a  low  probability  of  occurrence  and  low  mission  impact  and  decided 
not  to  remove  the  computer  from  the  spacecraft  to  resolve  the  issue.  We 
estimate  that  if  the  contractor  continues  to  perform  as  it  has  through  fiscal 
year  2008,  the  work  under  the  contract  at  completion  in  September  2011 
could  exceed  its  budgeted  cost  of  $1.6  billion  by  between  $621.7  million 
and  $1.2  billion. 


Page  85 


GAO-09-338  Defense  Acquisitions 


Appendix  II:  BMDS  Prime  Contractors  Exceed 
Budgeted  Cost  and  Schedule  Performance 
during  Fiscal  Year  2008 


Targets  and 
Countermeasures 
Program’s  Rebaseline 
Positively  Affected 
Fiscal  Year  Schedule 
Variances 


In  June  2008,  a  delivery  order  under  the  Targets  and  Countermeasures’ 
element  that  is  developing  a  new  family  of  targets — the  Flexible  Target 
Family  (FTF) — performed  a  rebaseline  as  a  result  of  experiencing 
manufacturing  delays  to  several  components.  The  majority  of  the  delays 
were  from  qualification  failures,  subsequent  redesigns,  and  requalification 
efforts.  The  rebaseline  was  to  realign  the  work  under  the  contract  to 
reflect  realistic  hardware  delivery  dates.  This  rebaseline  did  not  affect  cost 
variances,  but  did  rebaseline  major  milestone  delivery  dates  and,  as  a 
result,  set  some  of  the  previously  existing  schedule  variances  to  zero. 


The  Targets  and  Countermeasures  contractor  made  gains  with  a  favorable 
$23.2  million  fiscal  year  schedule  variance  due  in  part  to  the  rebaseline  in 
June  2008.  However,  the  contractor  ended  the  year  with  an  unfavorable 
cumulative  schedule  variance  of  $6.4  million  which  was  primarily  driven 
by  delays  in  the  completion  of  the  FTF  qualification  program.4  The 
program  also  ended  the  year  with  a  cumulative  cost  variance  of  $62.8 
million  which  the  contractor  attributed  to  costs  associated  with  the  FTF’s 
avionics  components  integration  and  qualification  issues,  and  more  effort 
than  expected  required  on  motors  for  one  of  the  targets  in  the  program. 
See  figure  18  below  for  an  illustration  of  cumulative  cost  and  schedule 
variances  during  the  course  of  the  fiscal  year. 


4  MDA  is  developing  the  FTF  to  represent  evolving  threats  of  all  ranges.  MDA  has  narrowed 
its  FTF  development  efforts,  focusing  on  a  single  vehicle,  the  72-inch  LV-2  ground-launched 
target. 
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Figure  18:  Targets  and  Countermeasures  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation). 


The  contractor  attributes  its  unfavorable  fiscal  year  cost  variances  of  $35.7 
million  to  the  increased  cost  of  completing  the  first  four  FTF  72-inch 
targets.  Delays  in  completing  component  qualification  extended  the  period 
of  performance  which  invariably  led  to  higher  costs.  In  addition,  the 
contractor  cites  cost  increases  to  the  failure  of  one  of  its  targets  in  July 
2008  that  added  mission  assurance  and  testing  cost  to  the  follow-on 
mission  using  the  same  target  configuration.  We  estimate  that  if  the 
contractor  continues  to  perform  as  it  has  in  the  past,  it  will  overrun  its 
budgeted  cost  of  $1.1  billion  at  contract’s  end  in  December  2009  by 
between  $63.7  million  and  $75.9  million. 


THAAD  Contractor 
Spent  More  Money 
and  Time  Than 
Budgeted 


The  Terminal  High  Altitude  Area  Defense  (THAAD)  program  experienced 
target  issues  during  fiscal  year  2008.  The  THAAD  program  performed  a 
realignment  in  May  2008  to  extend  the  flight  test  program  after 
experiencing  several  delayed  target  deliveries.  Because  of  the  cost  impact 
of  these  delayed  targets,  the  program  will  increase  its  value  of  the  work 
under  its  contract  by  approximately  $80  million.  The  THAAD  program 
performed  a  similar  realignment  in  December  2006  as  a  result  of  delayed 
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target  deliveries  as  well.  As  a  result  of  this  realignment,  the  program 
extended  its  flight  test  program  and  added  an  estimated  $121  million  to  the 
value  of  work  under  its  contract. 

The  THAAD  contractor  experienced  downward  trends  in  its  cost  and 
schedule  performance  during  fiscal  year  2008.  The  program  overran  its 
budgeted  costs  for  the  fiscal  year  by  $33.5  million.  It  was  also  unable  to 
accomplish  $7.4  million  worth  of  work  during  the  fiscal  year.  Both  of  these 
unfavorable  variances  added  to  the  negative  cumulative  cost  and  schedule 
variances  of  $228.7  million  and  $16.5  million,  respectively,  as  shown  in 
figure  19. 


Figure  19:  THAAD  Fiscal  Year  2008  Cost  and  Schedule  Performance 
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Source:  Contractor  (data),  GAO  (presentation). 


The  THAAD  prime  contractor’s  fiscal  year  cost  overrun  of  $33.5  million 
was  primarily  caused  by  the  radar,  missile,  and  launcher  portions  of  work. 
Design  problems  delayed  the  prime  power  unit  design  review  and  slowed 
parts  production,  causing  the  radar’s  negative  cost  trend.  In  addition,  the 
missile’s  negative  cost  trend  for  this  same  period  was  driven  by  design 
complexity,  ongoing  rework/retest  of  subsystems,  unexpected 
qualification  discoveries,  and  unfavorable  labor  variances  at  key 
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subcontractors.  Lastly,  the  launcher  variances  were  driven  by  hardware 
and  software  complexities  and  higher-than-expected  costs  for 
transitioning  a  portion  of  this  effort  to  a  different  facility  for  production. 

The  contractor  reports  that  its  unfavorable  fiscal  year  schedule  variance  of 
$7.4  million  is  primarily  driven  by  the  radar  and  missile  components.  The 
radar’s  negative  schedule  variance  is  associated  with  vendor  delays  in 
delivering  trailers  for  both  of  the  system’s  prime  power  units.  The  late 
delivery  of  the  trailers  has  subsequently  delayed  delivery  of  the  prime 
power  units.  Missile  rework  due  to  qualification  test  discoveries  also 
negatively  affected  schedule  performance.  If  the  contractor  continues  to 
perform  as  it  has  through  fiscal  year  2008,  we  project  that  at  the  contract’s 
completion  currently  scheduled  for  September  2009,  the  contractor  could 
overrun  its  budgeted  cost  of  $4.6  billion  by  between  $252  million  to  $274 
million. 
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On  May  23,  2008,  the  Senate  Armed  Services  Committee  requested  that  we 
review  the  reasons  behind  the  cancellation  of  a  GMD  flight  test  designated 
FTG-04.  Initially,  on  May  1,  2008,  the  Director,  MDA  decided  to  delay  this 
test  due  to  problems  discovered  in  a  telemetry  device,  the  Pulse  Code 
Modulation  Encoder  (PCME).  This  device  does  not  affect  operational 
performance,  but  rather  is  a  critical  component  needed  to  transmit  flight 
test  data  only.  The  PCME  problems  were  due  in  large  part  to 
manufacturing  defects,  which  the  manufacturers  and  MDA  concluded 
likely  affected  all  the  PCMEs.  However,  on  May  8,  the  Director  of  MDA 
instead  decided  to  cancel  this  flight  test  entirely,  resulting  in  one  less  GMD 
end-to-end  intercept  flight  test.  MDA  told  us  that  delaying  the  flight  test 
until  the  PCMEs  could  be  repaired  would  cause  delays  in  future  tests  since 
various  test  assets  were  shared.  MDA  officials  therefore  decided  to  cancel 
FTG-04  and  transfer  some  test  objectives  to  other  tests,  including  a  new 
non-intercept  flight  test,  FTX-03,  and  an  already  planned  intercept  flight 
test,  FTG-05.  Also,  for  some  remaining  objectives  not  captured  in  FTG-05 
and  FTX-03,  MDA  stated  that  it  planned  a  third  intercept  test,  FTG-X.  We 
were  asked  to  investigate  this  test  cancellation  and  answer  the  following 
questions: 

•  Why  did  the  MDA  change  its  initial  decision  to  delay  FTG-04  until 
November  2008  and  decide  to  cancel  FTG-04  instead  and  what 
deliberative  process  did  MDA  follow  in  deciding  to  cancel  FTG-04? 

•  When  and  how,  if  ever,  will  each  of  the  specific  test  objectives 
previously  planned  for  FTG-04  be  accomplished? 

•  What  are  the  implications  of  canceling  this  flight  test  on  the  ensuing 
test  program,  on  demonstrating  the  capability  of  the  GMD  system,  and 
on  other  programmatic  decisions? 


Faulty  Telemetry 
Component  Caused 
Delay  and  Subsequent 
Cancellation  of  FTG- 
04 


MDA  initially  delayed  the  FTG-04  flight  test  because  of  defects  in  the 
PCME,  a  telemetry  component  in  the  Exoatmospheric  Kill  Vehicle  (EKV) 
only  needed  to  gather  test  data.  Although  the  PCME  does  not  affect 
operational  performance,  it  is  needed  for  test  assets  to  determine  if  design 
and  operational  issues  have  been  resolved.  The  FTG-04  had  four  prior 
delays  and  was  originally  scheduled  for  the  first  quarter  of  fiscal  year  2007. 
In  responding  to  these  delays,  multiple  tests  over  several  years  were 
affected. 


Several  defects  contributed  to  the  problem,  the  first  three  of  which  are 
presumed  to  affect  all  PCMEs  manufactured  up  to  that  point  and  all  24 
fielded  Test  Bed/Capability  Enhancement  (CE)  I  EKVs: 

•  The  PCMEs  experienced  gold  embrittlement  due  to  lack  of  pretinning. 
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•  Insufficient  oscillator  stand-off  height  increased  thermal  stress. 

•  Circuit  board  deflection  caused  by  three  washers  missing  from  the 
board. 

In  addition  to  these  manufacturing  defects,  there  were  stress  fractures  in 
the  solder  of  three  PCMEs  caused  by  the  removal  and  replacement  of  a 
chip  on  the  device.  This  chip  was  removed  because  a  clock  on  a  chip  was 
asynchronous  with  another  component’s  clock.  It  was  estimated  that  there 
was  an  18  to  48  percent  chance  of  the  loss  of  telemetry  data  at  some  point 
during  a  flight  test  due  to  the  asynchronous  chip  problem.  Again,  all  24 
fielded  Test  Bed/CE-I  EKVs  have  the  chip  with  this  problem.  This  chip 
does  not  affect  operational  performance,  but  rather  is  a  critical  component 
needed  to  transmit  flight  test  data  only. 

See  table  10  for  timeline  of  events  related  to  this  cancellation. 


Table  10:  Timeline  of  Events 

1/12/08  -2/4/08 

During  early  tests  of  the  Payload  33  PCME  at  the  subcontractor's 
facility,  no  failures  were  detected. 

2/7/08 

During  final  test  readiness  reviews  at  Vandenburg  Air  Force  Base, 
the  first  failure  was  identified. 

2/8/08-2/22/08 

Trouble  shooting  isolates  problem  to  PCME  and  EKV  is  returned 
to  contractor  for  removal  of  PCM  E . 

2/22/2008 

MDA  de-emplaced  interceptor  as  a  replacement  (Payload  32). 

3/3/08-3/26/08 

Troubleshooting  continues,  failures  are  repeatable  but  intermittent. 

3/28/08-4/2/08 

Fault  isolated  to  oscillator  and  solder  joints. 

4/30/08-5/1/08 

Tiger  Team  formed  to  assess  risk  and  presented  risk  assessment 
to  the  Director,  MDA. 

5/1/08 

Director,  MDA  delays  FTG-04  test. 

5/8/08 

Program  Change  Board  recommends  cancellation:  Director,  MDA 
makes  the  decision  to  cancel,  replace  it  with  FTX-03  and  informs 
Congress. 

Source:  GAO  presentation;  MDA  documentation 


The  contractor,  Boeing,  and  the  subcontractors,  Raytheon  and  the 
manufacturer  of  the  component,  L-3,  took  actions  to  mitigate  the  problem. 
They  eliminated  the  gold  embrittlement  problem  by  sending  the  oscillator 
out  for  pretinning,  they  designed  custom  washers  for  two  already 
produced  PCMEs  and  raised  three  bosses  for  new  PCMEs  to  eliminate  the 
need  for  washers,  and  they  tightened  tolerances  on  the  board  to  eliminate 
the  deflection  issue.  These  first  three  PCME  manufacturing  improvements 
were  finalized  on  May  16,  2008.  They  also  made  changes  to  correct  the 
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chip  with  the  asynchronous  clock  problem  in  all  newly  manufactured 
PCMEs.  None  of  the  previously  fielded  GBIs  will  be  refurbished  with 
improved  PCMEs  needed  for  flight  tests,  but  the  GBIs  emplaced  starting  in 
October  2008  and  thereafter  have  the  improved  PCME. 

On  May  1,  2008,  MDA’s  Program  Change  Board  considered  five  options. 

1.  Execute  FTG-04  as  scheduled,  using  payload  (32)  “as  is” 

2.  Continue  diagnostic  testing  of  payload  32,  but  if  decision  was  made 
that  it  was  not  ready,  substitute  payload  33,  leading  to  a  delay  in 
the  test  schedule 

3.  Refurbish  payload  32,  but  if  it  did  not  improve,  substitute  payload 

33 

4.  Immediately  replace  payload  32  with  33  without  further  testing 

5.  Immediately  return  payload  32  for  repair 

The  Director,  MDA  chose  option  5,  delaying  the  FTG-04  into  the  November 
to  December  2008  timeframe,  but  keeping  the  program  on  track  to  provide 
this  intercept  data  as  planned.  MDA  consulted  the  test  community, 
Director,  Operational  Test  and  Evaluation  (DOT&E)  and  the  BMDS 
Operational  Test  Agency,  on  this  initial  decision  to  delay  the  test  and  both 
agreed  with  this  decision.  According  to  MDA,  the  Director  also  asked  for 
options  for  a  sensor  test  in  the  summer  of  2008. 

On  May  8,  2008,  MDA’s  Program  Change  Board  reconvened  to  consider 
three  options  for  a  sensor  test  (FTX-03)  and  canceling  instead  of  delaying 
FTG-04: 

1.  Conduct  FTX-03,  with  a  baseline  like  the  planned  FTG-04,  but 
without  a  live  intercept  attempt. 

2.  Conduct  FTX-03  with  a  baseline  like  the  FTG-05,  an  intercept  flight 
test  to  be  conducted  in  December  2008. 

3.  Similar  to  option  2,  but  with  more  sensor  data  collected. 

The  Director  MDA  changed  the  May  1  decision  to  delay,  refurbish  and  fly 
the  planned  FTG-04  test  and  chose  instead  to  cancel  FTG-04  and  pursue 
the  modified  option  3  above.  Choosing  option  3  resulted  in  restructuring 
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the  intercept  test  into  a  test  designed  to  assess  multiple  sensor  integration 
capability.  This  new  test  benefited  sensor  modeling  and  simulation  as 
post-flight  reconstruction  could  occur  now  on  two  missions. 

According  to  MDA,  it  canceled  the  FTG-04  at  the  May  8,  2008  meeting 
instead  of  delaying  it,  in  part,  because  rescheduling  FTG-04  would  have 
caused  a  major  delay  in  another  test,  the  Distributed  Ground  Test-03 
(GTD-03).  GTD-03  and  FTG-04  required  many  of  the  same  assets,  so 
conducting  FTG-04  would  have  delayed  GTD-03  and  thus  the  delivery  of 
this  new  capability  by  four  months.  Ground  tests  assess  the  increased 
BMDS  capability  to  be  fielded  next  and  GTD-03  was  to  provide  the  means 
by  which  a  more  realistic  simulation  of  threats  and  scenarios  and  the 
means  by  which  new  software  capability  could  be  declared  ready  to  move 
into  the  operational  baseline.  MDA  consulted  with  BMDS  Operational  Test 
Agency  officials  on  this  decision  and  they  supported  it.  DOT&E  was  not 
consulted  on  this  decision  and  expressed  concern  that  the  elimination  of 
any  intercept  test  reduced  the  opportunity  to  gather  additional  data  that 
might  have  increased  confidence  in  models  and  simulations.  DOT&E  has 
repeatedly  expressed  concerns  over  the  lack  of  test  data  needed  to 
validate  MDA’s  models  and  simulations. 


Most  FTG-04  Test 
Objectives  Will  Be 
Allocated  to  Follow- 
on  Tests 


According  to  MDA,  all  FTG-04  test  objectives  were  allocated  to  other  flight 
tests.  However,  partly  due  to  differences  in  how  MDA  describes  test 
objectives,  it  is  unclear  whether  all  planned  FTG-04  test  objectives  will  be 
accomplished  in  follow-on  tests.  The  loss  of  a  primary  objective,  an 
intercept  of  a  complex  target  scene,  will  slow  MDA’s  efforts  to  build 
confidence  in  the  EKV’s  ability  to  consistently  achieve  intercepts,  unless 
an  additional  intercept  is  scheduled.  In  August  2008,  MDA  informed 
Congress  that  it  planned  to  conduct  a  new  intercept  test  called  FTG-X  in 
fiscal  year  2009.  However,  in  January  2009  MDA  stated  that  the  FTG-X 
intercept  test  was  never  formally  approved  and  is  no  longer  planned. 


In  addition,  some  test  objectives  related  to  modeling  and  simulations  have 
been  redefined  so  it  is  unclear  whether  they  will  be  fully  tested.  Models 
and  simulations  are  critical  to  understanding  and  assessing  the 
performance  of  the  BMDS  because  flight  tests  are  limited  by  their  cost, 
complexity,  and  range  safety  constraints.  Modeling  and  simulation  is 
therefore  the  primary  way  to  fully  assess  the  overall  performance  of  the 
BMDS  and  its  various  components.  According  to  DOT&E,  cancellation  of 
FTG-04  reduced  interceptor  and  EKV  data  available  for  modeling,  leaving 
only  two  intercepts  (FTG-3a  and  FTG-05)  that  have  provided  complete  sets 
of  information.  In  October  2008,  MDA  stated  that  modification  of  FTG-04 
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into  a  sensor  test  eliminated  a  second  opportunity  to  anchor  the  models  of 
EKV-fielded  software.  Test  objectives  in  MDA  planning  documents 
describe  modeling  and  simulation  objectives  at  a  high  level.  However,  it  is 
difficult  to  determine  whether  modeling  and  simulation  objectives  are 
addressed  in  the  near  term  because  the  objectives  are  defined  differently 
for  each  test.  For  example,  the  FTX-03  and  FTG-05  objectives  do  not 
distinguish  between  primary  and  secondary  objectives  while  the  FTG-04 
does.  One  objective  that  seems  to  be  absent  in  the  FTG-05  and  FTX-03  is  to 
collect  data  to  support  validation  and  anchoring  of  system-level  (vs. 
element-level)  simulations,  MDA  stated  that  the  exclusion  from  the  test 
objectives  was  inadvertent  and  it  will  be  addressed  by  the  tests. 

BMDS  Operational  Test  Agency  objectives  related  to  the  GBI  engagement 
were  not  met,  although  the  test  agency  officials  indicate  the  majority  of 
their  non-intercept  sensor  related  objectives  for  FTG-04  were  met  in  the 
FTX-03  test.  However,  BMDS  Operational  Test  Agency  officials  state  that 
some  of  their  intercept  objectives  may  be  addressed  through  a 
combination  of  previous  intercept  test,  FTG-03a,  and  recently  conducted 
FTG-05.  Finally,  several  warfighter  objectives  for  FTG-04,  related  to 
tactics,  techniques,  and  procedures,  will  be  met  through  the  ground  tests 
instead  because,  according  to  the  warfighter  representative  at  the  BMDS 
Operational  Test  Agency,  flight  tests  do  not  offer  the  best  opportunity  to 
assess  this  kind  of  objective. 


Cancellation 
Eliminates  One  of 
Few  Opportunities 
Demonstrate  GMD 
Capabilities 


The  cancellation  has  increased  the  strain  on  the  ensuing  test  program. 
GMD’s  current  plans  call  for  two  intercept  attempts  in  fiscal  year  2009 — 
FTG-05,  which  was  conducted  in  December  2008,  and  FTG-06 — and  one 
booster  verification  test.  This  is  an  ambitious  schedule  as  GMD  has  been 
able  to  conduct  only  one  intercept  flight  test  per  year — FTG-02  in 
September  2006,  FTG-03a  in  September  2007  and  FTG-05  in  December 
2008.  MDA  had  planned  to  conduct  five  intercept  tests  with  varying 
stresses  to  assess  the  EKV  capability  between  February  2007  and 
December  2008.  Flight  test  failures  and  test  plan  revisions  caused  MDA  to 
only  carry  out  two  intercept  tests  in  that  period — FTG-03a  and  FTG-05 — 
both  of  which  resulted  in  an  intercept. 


In  addition,  the  number  of  future  flight  tests  planned  has  been  reduced. 
MDA  has  not  funded  or  scheduled  an  intercept  replacement  for  FTG-04.  In 
January  2008  MDA  decided  to  merge  two  intercept  tests — FTG-06  and 
FTG-07 — into  one  single  intercept  attempt.  This  merger  removes  another 
opportunity  to  gather  end-game  EKV  performance  data  needed  to  assess 
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capability.  In  January  2008  MDA  also  decided  to  accelerate  a  two-stage 
verification  non-intercept  test  required  to  assess  the  European  component. 

The  cancellation  of  FTG-04  removed  one  chance  to  obtain  end-game 
performance  data  needed  to  develop  GMD  models  and  to  assess  the 
capability  of  the  CE-I  EKV.  The  repetition  of  intercept-related  objectives  is 
important  to  build  confidence  in  the  intercept  capability.  These  models  are 
the  primary  way  to  fully  assess  the  overall  system  performance,  since 
flight  tests  are  limited  by  their  cost,  complexity  and  range  safety  concerns. 
MDA  planned  to  test  the  CE-I  EKV  against  a  dynamic  target  scene  with 
countermeasures  in  both  FTG-04  and  FTG-05.  FTG-04  was  canceled  and  an 
FTG-05  target  anomaly  affected  this  objective.  According  to  MDA,  no  more 
CE-I  EKV  flight  tests  have  been  approved,  although  it  is  considering 
whether  to  conduct  an  intercept  test  using  a  CE-I  EKV  in  the  future.  GMD 
developed  some  mitigations  to  various  developmental  issues,  but  realistic 
flight  testing  is  needed  to  anchor  the  models  and  to  determine  the 
effectiveness  of  these  mitigations. 

The  test  cancellation  and  target  problems  have  reduced  the  knowledge 
that  MDA  expected  to  use  for  its  upcoming  end-to-end  performance 
assessment.  Performance  assessments  are  annual  system-level 
assessments  to  test,  evaluate,  and  characterize  the  operational  capability 
of  the  BMDS  as  of  the  end  of  the  calendar  year.  Currently,  MDA  has  only 
completed  one — Performance  Assessment  2007.  Furthermore,  acting  on  a 
joint  recommendation  between  MDA  and  the  Operational  Test  Agency, 
MDA  officials  canceled  their  2008  performance  assessment  efforts  in  April 
2008  because  of  developmental  risks  associated  with  modeling  and 
simulations.  Instead,  MDA  is  focusing  on  testing  and  models  for 
Performance  Assessment  2009.  However,  the  planned  performance 
information  available  for  Performance  Assessment  2009  will  be  reduced. 
The  FTG-04  cancellation  reduced  one  set  of  data  that  was  expected  to  be 
available.  In  addition,  both  FTX-03  and  FTG-05  will  be  used  to  anchor  data 
for  Performance  Assessment  2009,  but  target  anomalies  in  each  test 
precluded  the  completion  of  all  planned  test  objectives.  Neither  target 
presented  the  complexity  needed  for  advanced  algorithm  development. 

Manufacturing  and  emplacement  continue  unabated  by  reductions  and 
delays  in  tests.  Twenty-four  CE-I  GBIs  have  been  fielded  and  the  new  CE-II 
GBIs  are  now  being  fielded  without  important  knowledge  about  the 
systems  capabilities  expected  to  be  gained  through  tests.  The  first  CE-II 
GBI  emplacement  occurred  prior  to  any  flight  testing  of  this  configuration. 
The  first  flight  test  is  FTG-06  currently  scheduled  to  occur  no  earlier  than 
the  fourth  quarter  of  fiscal  year  2009.  According  to  MDA,  these  CE-II  GBIs 
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will  not  be  declared  operational  until  after  the  successful  completion  of 
FTG-06.  FTG-04  was  also  identified  as  a  key  source  of  data  supporting  a 
number  of  capabilities  declarations.  The  cancellation  of  FTG-04,  plus  other 
testing  delays,  prompted  MDA  to  defer  some  capabilities  and  to  declare 
others  based  on  previous  tests. 


Conclusions 


The  cancellation  of  the  FTG-04  flight  test  increases  the  risk  to  the  GMD 
program  and  to  the  overall  BMDS  capability,  since  the  lack  of  adequate 
intercept  data  adversely  affects  confidence  that  the  system  could  perform 
as  intended  in  a  real-world  situation.  The  GMD  program  has  reduced  its 
plans  to  assess  operational  performance  of  the  fielded  configuration 
between  February  2007  and  December  2008  from  five  to  two  intercept 
tests,  leaving  gaps  in  knowledge  about  the  repeatability  of  the 
performance  of  fielded  assets.  In  addition,  the  opportunity  to  obtain 
additional  intercept  data  vital  to  the  anchoring  of  models  and  simulations 
has  been  lost,  unless  the  FTG-X  flight  test  is  conducted,  adding  to  an 
existing  concern  expressed  by  DOT&E.  Despite  test  reductions  and  effects 
on  assessing  system-level  performance,  production  and  fielding  of  assets 
continues  as  planned. 
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The  two  tables  below  list  test  events  supporting  MDA  capability 
declarations  during  fiscal  year  2008  for  certain  engagement  sequence 
groups  in  Blocks  1.0  through  3.0  (see  table  11),  as  well  as  for  the  full 
completion  of  Bock  1.0  by  the  end  of  fiscal  year  2009  (see  table  12).  Both 
tables  illustrate  that  MDA  reduced  the  basis  for  declaring  certain 
engagement  sequence  groups  as  early  or  fully  capable.  The  basis  for 
declaring  an  early,  partial,  or  full  capability  includes  flight  and  ground 
tests  as  well  as  performance  assessments. 

Table  1 1 :  Engagement  Sequence  Groups  with  Revised  Basis  for  Fiscal  Year  2008  Capability  Declarations 

Engagement  sequence 

Capability  declaration 

Planned  basis 
for  2008  declaration 

Revised  basis 
for  2008  declaration8 

Block  1.0 

GBI  Launch  on  COBRA  DAN  E/Upgraded  Early  Early 

PA  07 

Dropped 

warning  Kaaar 

GTD-02 

V 

GTI-02 

V 

GTG-04-3 

GBI  Engage  on  Sea-based  X-band  Radar  Early 

PA  07 

Dropped 

GTD-02 

V 

GTI-02 

V 

GTG-04-3 

GBI  Launch  on  Sea-based  X-band  Radar  Early 

PA  07 

Dropped 

GTD-02 

V 

GTI-02 

V 

GTG-04-3 

Block  2.0 

SM-3  Engage  on  shipboard  Aegis  Radar  Full 

PA  08 

Dropped 

FTM-15 

Dropped 

GTD-03 

Dropped 

GTI-03 

Dropped 

GTX-033 

Dropped 

FTM-10,-11,-12,-13 

GTD-02 

GTI-02 

SM-3  Launch  on  Remote  shipboard  Aegis  Early 

_  _l  _ 

GTD-02 

Dropped 

Kaaar 

GTI-02 

Dropped 

FTM-13 

Dropped 

FTM-14 
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Engagement  sequence 

Capability  declaration 

Planned  basis 
for  2008  declaration 

Revised  basis 
for  2008  declaration8 

THAAD  Engage  on  AN/TPY-2  (terminal  mode) 

Early 

FTT-09 

V 

FTT-07,-08 

GTI-02, -03 

GTX-03a 

Block  3.0 

GBI  Launch  on  shipboard  Aegis  Radar 

Early 

GTX-038 

Dropped 

GTD-02 


GTI-02 

GTI-03 

FTX-03 


Source:  GAO  analysis  of  MDA  data. 

“Planned  assessment  or  test  was  actually  used  for  the  capability  declaration  indicated  by  an  "V"  in 
this  column. 
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Table  12:  Block  1.0  Engagement  Sequence  Groups  with  Revised  Basis  for  Completion  at  End  of  Fiscal  Year  2009 

Engagement  sequence  Capability  declaration 

Planned  basis 
for  2008  declaration 

Revised  basis 
for  2009  declaration8 

GBI  Engage  on  COBRA  DANE  Full 

(Beale  Air  Force  Base,  CA) 

PA-07 

Dropped 

GTD-02 

V 

GTI-02 

V 

FTG-04 

Dropped 

GBI  Launch  on  COBRA  DANE  (Beale  Air  Force  Partial 

Base,  CA) 

GTD-03 

Dropped 

GTI-03 

V 

FTG-04 

Dropped 

GTD-02 

FTI-02 

Full 

PA-08 

PA-09  Quick  Look 

GTD-03 

V 

GTI-03 

V 

FTG-05 

Dropped 

GBI  Engage  on  Sea-based  X-band  radar  Partial 

GTD-03 

Dropped 

GTI-03 

V 

Full 

PA-08 

PA-09  Quick  Look 

GTD-03 

V 

GTI-03 

V 

FTG-05 

Dropped 

GBI  Launch  on  Sea-based  X-band  radar  Partial 

GTD-03 

Dropped 

GTI-03 

V 

FTG-04 

Dropped 

Full 

PA-08 

PA-09  Quick  Look 

GTD-03 

V 

GTI-03 

V 

FTG-05 

Dropped 

GBI  Engage  on  forward-based  AN/TPY-2  radar  Full 

PA  07 

Dropped 

GTI-03 

GTD-02 

V 

GTI-02 

V 

FTG-04 

Dropped 

FTX-03 
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Engagement  sequence 

Capability  declaration 

Planned  basis 
for  2008  declaration 

Revised  basis 
for  2009  declaration8 

GBI  Launch  on  forward-based  AN/TPY-2  radar  Full 

PA  07 

Dropped 

GTI-03 

GTD-02 

V 

GTI-02 

V 

FTG-04 

Dropped 

FTX-03 

GBI  Engage  on  shipboard  Aegis  radar 

Full 

PA  07 

Dropped 

GTI-03 

GTD-02 

Dropped 

GTI-02 

Dropped 

FTG-04 

Dropped 

FTX-03 

GBI  Launch  on  shipboard  Aegis  Radar 

Full 

PA  07 

Dropped 

GTI-03 

GTD-02 

Dropped 

GTI-02 

Dropped 

FTG-04 

Dropped 

FTX-03 

Source:  GAO  analysis  of  MDA  data. 

“Planned  assessment  or  test  for  the  capability  declaration  that  hasn't  changed  is  indicated  by  an  V" 
in  this  column. 
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To  examine  the  progress  MDA  made  in  fiscal  year  2008  toward  its  cost, 
schedule,  testing,  and  performance  goals,  we  examined  the  efforts  of  10 
BMDS  elements  that  MDA  is  developing  and  fielding.  The  elements 
included  in  our  review  collectively  accounted  for  80  percent  of  MDA’s 
fiscal  year  2008  research  and  development  budget  requests.  In  assessing 
each  element,  we  examined  the  BMDS  Fiscal  Year  2008  Statement  of 
Goals,  Program  Execution  Reviews,  test  plans  and  reports,  production 
plans,  Contract  Performance  Reports,  MDA  briefings,  and  earned  value 
management  data.  We  developed  data  collection  instruments  that  were 
completed  by  MDA  and  each  element  program  office.  The  instruments 
gathered  detailed  information  on  planned  and  completed  program 
activities  including  tests,  design  reviews,  prime  contracts,  estimates  of 
element  performance,  and  challenges  facing  the  elements.  In  addition,  we 
discussed  fiscal  year  2008  progress  and  performance  with  officials  in 
MDA’s  Agency  Operations  Office,  each  element  program  office,  as  well  as 
the  Office  of  DOD’s  Director,  Operational  Test  and  Evaluation,  and  DOD’s 
Operational  Test  Agency.  To  assess  each  element’s  progress  toward  its 
cost  goals,  we  reviewed  Contract  Performance  Reports  and,  when 
available,  the  Defense  Contract  Management  Agency’s  analyses  of  these 
reports.  We  applied  established  earned  value  management  techniques  to 
data  captured  in  Contract  Performance  Reports  to  determine  trends  and 
used  established  earned  value  management  formulas  to  project  the  likely 
costs  of  prime  contracts  at  completion. 

To  evaluate  the  sufficiency  of  MDA’s  modeling  and  simulation  practices, 
we  reviewed  DOD  and  MDA  policies,  memos,  flight  and  test  plans  related 
to  modeling  and  simulations,  the  Acquisition  Modeling  and  Simulation 
Master  plan,  as  well  as  verification,  validation  and  accreditation  plans  and 
reports  for  various  elements,  and  MDA  white  papers  discussing  modeling 
and  simulation  techniques.  We  also  interviewed  officials  in  element 
program  offices  to  discuss  modeling  and  simulation  plans  and  procedures 
particular  to  each. 

In  assessing  MDA’s  accountability,  transparency,  and  management 
controls,  we  interviewed  officials  from  the  Office  of  the  Under  Secretary 
of  Defense’s  Office  for  Acquisition,  Technology,  and  Logistics,  as  well 
officials  in  the  MDA  Agency  Operations  Directorate.  We  also  reviewed  an 
Institute  for  Defense  Analysis  study,  two  Congressional  Research  Service 
reports,  a  Congressional  Budget  Office  report,  U.S.  Code,  DOD  acquisition 
system  policy,  various  DOD  directives,  the  Missile  Defense  Executive 
Board  charter,  and  various  MDA  statements  and  documents  related  to  the 
agency’s  block  structure. 
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To  ensure  that  MDA-generated  data  used  in  our  assessment  are  reliable, 
we  evaluated  the  agency’s  management  control  processes.  We  discussed 
these  processes  with  MDA  senior  management.  In  addition,  we  confirmed 
the  accuracy  of  MDA-generated  data  with  multiple  sources  within  MDA 
and,  when  possible,  with  independent  experts.  To  assess  the  validity  and 
reliability  of  prime  contractors’  earned  value  management  systems  and 
reports,  we  interviewed  officials  and  analyzed  audit  reports  prepared  by 
the  Defense  Contract  Audit  Agency.  Finally,  we  assessed  MDA’s  internal 
accounting  and  administrative  management  controls  by  reviewing  MDA’s 
Federal  Manager’s  Financial  Integrity  Report  for  Fiscal  Years  2003,  2004, 
2005,  2006,  2007,  and  2008. 

Our  work  was  performed  primarily  at  MDA  headquarters  in  Arlington, 
Virginia.  At  this  location,  we  met  with  officials  from  the  Aegis  Ballistic 
Missile  Defense  Program  Office;  Airborne  Laser  Program  Office; 

Command,  Control,  Battle  Management,  and  Communications  Program 
Office;  MDA’s  Agency  Operations  Office;  DOD’s  Office  of  the  Director, 
Operational  Test  and  Evaluation;  and  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics.  In  addition,  in 
Huntsville,  Alabama,  we  met  with  officials  from  the  Ground-based 
Midcourse  Defense  Program  Office,  the  Sensors  Program  Office,  the 
Terminal  High  Altitude  Area  Defense  Project  Office,  the  Kinetic  Energy 
Interceptors  Program  Office,  the  BMDS  Kill  Vehicles  Program  Office,  the 
Targets  and  Countermeasures  Program  Office,  and  the  Office  of  the 
Director  for  BMDS  Tests.  We  also  met  with  Space  Tracking  and 
Surveillance  System  officials  in  El  Segundo,  California. 

In  December  2007,  the  conference  report  accompanying  the  National 
Defense  Authorization  Act  for  Fiscal  Year  2008  noted  the  importance  of 
DOD  and  MDA  providing  information  to  GAO  in  a  timely  and  responsive 
manner  to  facilitate  the  review  of  ballistic  missile  defense  programs. 
During  the  course  this  audit,  we  experienced  significant  delays  in 
obtaining  information  from  MDA.  During  the  audit,  MDA  did  not  provide 
GAO  staff  with  expeditious  access  to  requested  documents  which  delayed 
some  audit  analysis  and  contributed  to  extra  staff-hours.  Of  the  documents 
we  requested,  we  received  approximately  19  percent  within  the  10-15 
business  day  protocols  that  were  agreed  upon  with  MDA.  Pre-existing 
documentation  took  MDA  on  average  about  50  business  days  to  provide 
and  many  pre-existing  documents  took  over  100  business  days  to  be 
provided  to  GAO.  Notwithstanding  these  delays,  we  were  able  to  obtain 
the  information  needed  to  satisfy  our  objectives  in  accordance  with 
generally  accepted  government  auditing  standards. 
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We  conducted  this  performance  audit  from  May  2008  to  March  2009  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives. 
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commitment  to  good  government  is  reflected  in  its  core  values  of 
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